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ABSTRACT 



The experiences of 11 innovative public policy 



education projects funded by the W. K. Kellogg Foundation between 
1988 and 1992 are reviewed. It presents results of a cluster 
evaluation that used a case study approach to examine effective 
public policy education and the role of coalitions. The report begins 
with project descriptions that include a summary of the issues 
addressed, the policy arena in which the issues would be decided, 
audience, and coalition membership. Projects are as follows: the 
Global Food Web, Iowa Public Policy Education Project, Agriculture 
and Food Policy in an Interdependent World, Restructuring the Upper 
Midwest, Northeast Network Project, Partners in Natural Resource 
Policy, Groundwater Public Policy Education Project, Policy Options 
and Strategies for Total Community Adjustment, Communicating 
America's Farm Policy, Trade and Development Program, and Food Forum 
Education Project. Findings are presented in the forms of 21 lessons. 
Each lesson is accompanied by a discussion of the evidence supporting 
it and its implications; many are further illustrated by vignettes 
drawn from individual projects. Three longer case studies of 
individual projects are included. The report concludes with 
reflections on projects' experiences in light of continuing 
developments in the practice of public policy education in these 
areas: coalition building, implementation of educational programs, 
and impacts of education about public issues. Appended is a 
description of the methodology. Contains 30 references. (YLB) 
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INTRODUCTION 



Our political system is in trouble. The most obvious 
symptom r- voter turnout, which, despite an upturn in 
1992, is much lower than it was in 1960. Thai trend has 
been accompanied by declining irusi in government and 
belief that public officials care what citizens think. Ai the 
same lime, growing concern — approaching anger — aboul 
persistent and multiplying public problems seems matched 
only by the inability of policymakers lo solve ihem. 
Policymakers seem trapped by special interests and absence 
of public support. Involvement in policy-making has 
narrowed to an "iron triangle" of legislators, bureaucrats, 
and interest groups that is increasingly polarized in ideo- 
logical ways (see, e.g.. Ginsberg and Shelter 1990; Dionnc 
1991). Any decision by policymakers produces costly 
enemies, and the public is not there to back anyone up. 

There is, however, a growing desire for change. Recent 
studies relying on focus groups or qualitative interviews 
(Sanders 1990; Harwood 1991 ) indicate that citizens arc 
tired ol negative campaigns and the politics of "sound 
bites." They say they want lo know more aboul issues and 
how they affect "me and the people 1 care aboul." People 
avoid politics partly because they see organizing and 
fighting as the only way to gel involved effectively, and they 
are uncomfortable with that style of politics. They under- 
stand that issues are complicated, and they want to know 
what the tradeoffs are. They want a policy-making process 
they can trust. They hope for polices that satisfy their self- 
interests, but tell interviewers that fairness to others is a 
more important standard for judging proposals and out- 
comes (Hochschild 1981; Sanders 1990). 

Clearly, people recognize the need to find better ways to 
make public policy decisions, and educators have a poten- 
tially important role to play. Cooperative Extension has 
recently adopted a national policy statement and "action 
agenda" in support ol increased education about public 
issues (Extension Committee on Organization and Policy 
and Extension Service l J SDA 1992; Cooperative Extension 
System Task Force on Public Issues Education 1993), and 
several state extension organizations have adopted similar 
statements. A series of national videoconferences on public 
issues education has been produced (Dale 1993). and new 
printed materials have been developed lo support in-service 
education (Public Issues Education Materials Task Force 
1904). Meanwhile, other organizations such as the 



Kettering Foundation, Study Circles Resource Center, and 
Project Public Life continue to develop and promote 
opportunities for education aboul public issues (McAfee, 
McKenzic, and Mathews n.d.; Project Public Life n.d.; 
Study Circles Resource Cenicr 1993). 

It thus seems timely to review the experiences of eleven 
innovative public policy education projects funded by the 
W. K. Kellogg Foundation in collaboration with ihe Farm 
Foundation between 1988 and 1992. The foundations' 
purpose in supporting ihe projects was to "strengthen or 
develop ongoing public policy education programs and 
involve more people and institutions in discussions of 
agricultural and rural issues" (W. K. Kellogg Foundation 
and Farm Foundation, "Opportunities for Innovative Public 
Policy Education Program Development," September 
1987). A key stipulation was that proposals had to come 
from a coalition of two or more organizations to "insure 
that the best informal ion available is brought to bear on 
policy decisions. . .help assure that a variety of perspectives 
is represented, [and]. . .increase the potential for continuity 
of ihe programs after the termination of Kellogg funding." 
Each project was required to conduct its own evaluation, 
bui the foundation also funded a separate, comparative, 
"cluster evaluation" lo look across ihe projects to see what 
could be learned aboul effective public policy education 
and the role of coalitions. This report is the result of thai 
evaluation. 

The cluster evaluation used a case study approach. We 
learned aboul the projects mainly from periodic site visils — 
relying heavily on interviews with project staff, coalition 
members, and participants in project events; observations 
of selected planning meetings and educational activities; 
and review of proposals, annual reports, evaluation studies, 
and other project documents. We sought, first, to under- 
stand the story of each project's genesis, implementation, 
challenges, and successes, and, second, via comparative 
analyses of these individual stories, to extract broader 
understandings of key elements of effective public policy 
education. (See the appendix for a more detailed discussion 
ol our methodology.) 

All dala weie collected using approved procedures for 
protecting the privacy of the human participants. Specifi- 
cally, all respondents were offered full information aboul 
the evaluation and their participation in it. were asked for 
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their voluntary participation, and were assured that their 
individual responses would be kept wholly confidential. At 
the stage of writing this report, all project directors were 
contacted and asked for permission to use actual organiza- 
tional names. All eleven projects granted permission lo do 
so. Nevertheless, individual identities remain confidential; 
all references to individuals employ pseudonyms, and most 
arc a composite portrait of several individuals. 

The report begins with a descriptive introduction to the 
eleven projects. Findings arc then presented in the form of 
twenty-one lessons. Each lesson is accompanied by a 
discussion of the evidence supporting it and its implica- 
tions. Man\' lessons are further illustrated by vignettes 
drawn from individual projects. Three longer case studies 
of individual projects arc also included. The report con- 
cludes with reflections on these projects experiences in 
light of continuing developments in the practice of public 
policy education. Finally, in the margins throughout the 
report, we offer a sampling of humorous and insightful 
quotations from our interviews. 
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THE PROJECTS 



The projects that formed ihc basis of our comparative 
evaluation are described below. Each description includes a 
summary of the issues addressed by the project, the policy 
arena in which the issues would be decided, the ewe/inur 
for the project, and the membership of the project s 
coalition. 

• The Global Food Web Project in Georgia developed a 
curriculum on human nutrition, world food supply, and 
the environment, emphasizing creative learning activi- 
ties. During each of the next two years volunteer 
instructors were trained, and a week-long statewide 
conference was held for county teams of lour teens and 
one adult leader. Following the conference, each team 
was expected to teach some of what they had learned to 
150 other individuals. 

Issue: the "global food web" — human nutrition, world 
food supply, and the environment 

Policy arena: international but with an expectation of 
local action 

Audience: teens from throughout the state 

Coalition: extension and a management council made 
up of extension specialists and agents, representatives ol 
other (nonextension) units at the University of Georgia 
(e.g.. Bureau of Educational Services. Torrencc Center 
lor Creative Studies. Institute of Ecology. Institute for 
Community and Area Development, Botanical Gardens, 
and international Development). and representatives ol 
other organizations (e.g.. State Department of Natural 
Resources, U.S. Environmental Protection Agency. Zoo 
Atlanta. Regional Education Service Agency, and State 
Department of Education) 

• The Iowa Public Policy Education Project (PPEP) 

used preference surveys ol stale legislators, interest 
groups, and extension councils and a statewide "agenda 
conference'" lo idenlifx priority issues and then con- 



ducted statewide "satellite town meetings" on each issue, 
covering two issues per year. A videotaped docu men tan- 
was developed for each issue in which multiple speakers 
laid out the issues, options, and local examples. Each 
satellite town meeting included the documentary, a live 
statewide panel, local panels at each downlink site, local 
discussion, and completion of preference surveys. (In 
the last year of the project, the documentaries were 
broadcast on commercial television.) In addition, 
statewide "focus groups" of emerging leaders e.v-'ored 
each issue in greater depth with a six-session program of 
speakers and field trips. Mini-grants to support focus 
groups at the local level were added during the last year 
of the project. 

Issue: the farm bill, drug abuse, waste management, 
health care, education, and the state budget crisis 

Policy arena: state (except for the farm bill), with 
increasing attention given to local implications as the 
project evolved 

Audience: citizens of the state (for the satellite town 
meetings): "emerging leaders" Ironi across the state (for 
the focus groups) 

Coalition: extension and an eighteen- mem her council, 
partly appointed by the university president and partly 
elected at the "agenda conference," plus representatives 
of the state legislature and extension field staff 

• Agriculture and Food Policy in an Interdependent 
World, an Illinois project, featured roundlable discus- 
sions of issues related lo the farm bill. Discussions were 
held ai lour locations throughout the state and involved 
farmers, environmentalists, and others with conflicting 
positions on ihc issues. The project also provided 
background materials be ore the roundlables and ml low- 
up newsletters on farm bill developments. A year later, a 
follow-up conference w as held with panels, ample time 
lor questions and answers, and discussion groups on one 



tor re gi«n£ to 1 1 'tool extension ior tin- next ^eneScitK-n iind 
dwig us ciggic*; into the twenty century. 

—project leader 
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of the topics. Finally, ihc project director helped write a 
"city person's guide to farm policy." 

Issue: the farm bill — specifically, international trade, 
commodity programs, environment, food programs, and 
food safety 

Policy arena: national 

Audience: farm and nonfarm citizens throughout the 
state 

Coalition: extension and the state League of Women 
Voters, with additional support from the Land of Lincoln 
Soybean Association 

• Restructuring the Upper Midwest, a project carried 
out in Minnesota. North Dakota. South Dakota, and 
Montana, used brainstorming sessions or surveys, 
variously implemented in irach state, to identify issues. 
Conferences were then sponsored at different locations, 
convenient for one or two states, with speakers and 
panels to clarify issues and outline possible solutions. As 
time went by. the project made increasing use. within 
the conferences, of discussion groups asked to identify 
policy recommendations. 

Issue: general — the need lor "restructuring" in a region 
undergoing population decline; specific — rural health 
care, rural education, rural economic development, and 
United States-Canada trade 

Policy arena: state, primarily, but with secondary 
attention given to local and nuiltistate arenas 

Audience: legislators, agency administrators, profession- 
als, and other slate and local leaders 

Coalition: extension in Minnesota. South Dakota, and 
North Dakota; the Montana Stale University Local 
Government Center; and the Pickrcl Seminars :u the 
University of Minnesota 

• I hc Northeast Network Project dev eloped educa- 
tional materials about the food system in the Northeast, 
consisting of live modules — an overview, one module 



each on food safety, food costs, and nutrition, and one 
on the international environment. The materials in- 
cluded background papers for each module, a summary 
and an overv iew of each paper, videos to accompany the 
modules, and a facilitators guide and accompanying 
video. Development of materials was followed by in- 
service education for extension staff in the thirteen 
northeastern states and support for local pilot projects. A 
desired outcome was the creation of local councils or 
networks ol diverse food system representatives. 

Issue: the food system in the northeastern United 
States — specifically, food safety, food costs, and nutrition 

Policy arena: local, state, national, and international 

Audience: individuals who play a role in a community s 
food system, such as suppliers, producers, processors, 
distributors, wholesalers, retailers, and consumers 

Coalition: extension in New York and Pennsylvania, 
with intended participation from extension in eleven 
other northeastern states 

• Partners in Natural Resource Policy, an Arizona 
project, focused on a single county and used a participa- 
tory approach in which staff conducted informal inter- 
views to idem if)' key players and concerns, conducted a 
telephone survey in English and Spanish to determine 
grass-roots concerns over issues, and developed events 
(a "riverwalk" and a conference) to address those 
concerns. 

Issue: water quality and supply 

Policy arena: local or. perhaps more accurately, regional 
(stressing connections among Nogalcs, Arizona; 
Nogales. Mexico; and Tucson) 

Audience: local citizens affected by water issues 

Coalition: extension alone, working toward a grass-roots 
coalition ol local citizens and agency representatives as 
well as a state-level coalition lor public policy education 



Oui flo.il wtis not to influence tin politic dl pun ess, bill to I ho [stdte] legisldtuie meets for ninety tKiys every two yedts. 

open it up. Some people believe it should meet lot two days every ninety 

-project leader years 

-coalition member 
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• The Groundwater Public Policy Education Project 
(CPEP) developed educational materials and then 
implemented seven state-level pilot projects. The 
educational materials included a special issue of the 
Journal of Soil and Water Conservation, seventeen leaflets 
on groundwater and public policy, and a handbook ior 
pilot project leaders (the latter was not completed). Each 
pilot project was different and had its own coalition, 
audiences, objectives, and strategics. Pilot project 
activities included statewide conferences, regional 
workshops for citizens and local leaders, development 
and presentation of county "water schools," support for 
local efforts to educate and develop groundwater policy, 
and efforts to infuse public policy education into current 
cdu ca t io nal p rogra m m i ng. 

Issue: groundwater quality 

Policy arena: state and local 

Audience: primarily suite and local elected officials and 
agency personnel, but some pilot projects also targeted 
citizens, water-related interest groups, researchers, and 
others 

Coalition: extension in California, Florida, Iowa. North 
Carolina. New York. Pennsylvania, and Wisconsin; the 
Freshwater Foundation: and the Soil and Water Conser- 
vation Society 

• Policy Options and Strategies for Total Commu- 
nity Adjustment, a Texas project, sponsored a state- 
wide rural development workshop to identify problt ms 
and options and engaged in follow-up work designed to 
create a more broadly based rural development coalition 
of legislators, agencies, and interest groups. At the local 
level, in two four-county regions, the project encouraged 
community leaders to cooperate across community lines 
to identify problems and rural development strategies. 

Issue: rural development, broadly defined 

Policy arena: state and local (multicommunity) 

Audience: state agencies and interest groups and local 
communiiv leaders 



Coalition: extension alone, with the objective of devel- 
oping a coalition of state agencies and interest groups 
and coalitions of community leaders from neighboring 
communities (an initial coalition with the Texas Bankers 
Association ended when the latter withdrew because of 
budgetary problems) 

Communicating America's Farm Policy produced 
press releases based on articles in Chokes, a policy- 
oriented magazine published by tnc American Agricul- 
tural Economics Association, and sponsored press 
conferences with authors of the articles. The project also 
sponsored informational "backgrounders" for members 
of the rural and urban press and experimented with 
other strategics for increasing the readership of Chokes, 
including radio spots and a speakers bureau. 

Issue: food, farm, and resource concerns 

Policy arena: varied with issue, but mostly national or 
international 

Audience: members of the rural and urban press and, 
ultimately, consumers 

Coalition' two professional associations, the American 
Agricultural Economics Association and the American 
Agricultural Editors Association 

The Trade and Development Program (TDP) 

developed educational materials on trade and develop- 
ment issues that comprised a kit for individual and 
group study; additional information was provided 
throughout the project. Various dissemination strategics 
were tried, including working through participating 
organizations in the projects coalition and holding 
locally based conferences on these issues. The project 
also sponsored an educational trip to Central America to 
stuth' trade issues. 

Issue: linkages among l\S. trade policies. Third World 
development, and U.S. agriculture 

Policy arena: international, although the project 
attempted to find local connections 



We hove made ti concerted effort to give the project aw<iy. 

-project leader, on the importance of local 
people having ownership of the project 



f oottwll seems to be the root of dll the problems. 

-project staff, noting the difficulty of promoting 
cooperation across local community boundaries 
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Audience: members of the participating organizations, 
expanded toward the end of the project to other grass- 
roots groups and individuals 

Coalition: nine national "farm and faith" groups — Bread 
for the World Institute on Hunger and Development, 
Center for Rural Affairs, Evangelical Lutheran Church in 
America, Friends Committee on National Legislation 
Education Fund, National Catholic Rural Life Center, 
National Fanners Organization, National Farmers 
Union, Presbyterian Church (USA), and U.S. Catholic 
Conference — forming an "oversight committee" com- 
posed of a representative from each organization (a tenth 
group, the National Grange, left the coalition after the 
project was under way) 

♦ The Food Forum Education Project developed 

educational events and materials on three issues, one per 
year for the duration of the project. Each year, the two 
collaborating organizations cosponsored a "food forum" 
roundtable discussion involving national leaders on 
different sides of food policy issues and then worked 
separately to hold leadership training workshops for 
their members and prepare publications for use in 
citizen education. In the third yean the forum and the 
two leadership training events were merged. In addition, 
six "pass-through" grants were awarded each year for 
model citizen education projects at the local level. 

Issue: pesticides, the farm bill, and sustainable agricul- 
ture 

Policy arena: mainly national, but becoming increas- 
ingly local during the course of the project 

Audience: players in national food policy issues and 
local citizens 

Coalition: National League of Women Voters Education 
Fund and Public Voice for Food and Health Policy (a 
consumer food and agriculture research, advocacy, and 
education organization) 

The experiences of these diverse projects suggested the 
following initial lessons: 



LESSON 1 

It is possible to implement successful-public policy 
education programs involving a diversity of issues, policy 
arenas, audiences, and project designs. 

A wide variety of issues was addressed by the cluster of 
eleven projects, including the 1990 farm bill, commodity 
programs, international trade, environmental protection, 
waste management, groundwater, nutrition, food safety, 
food supply, health care, education, economic develop- 
ment, and substance abuse. The policy arenas in which the 
issues were decided ranged from local to international. In 
the majority of cases, the issues to be addressed were 
selected by project leaders; in others, they were chosen by 
panels of experts, local leaders, or citizens, or by a demo- 
cratic process. The choice of issues was frequently influ- 
enced by the audiences the projects hoped to reach. There 
were notable differences in the degree to which coalitions 
or their dominant organizations selected issues outside 
their normal or traditional areas of expertise. 

Target audiences for the projects were nearly as varied as 
the issues, ranging from relatively small groups of state - 
level policy leaders to the general public; from relatively 
homogeneous groups, such as youth or the press, to 
heterogeneous ones varying in substantive expertise and 
perspectives on the issue; and from individuals relatively 
close to the policy process, such as national, state, or local 
legislators and agency personnel, interest group leaders, the 
press, and technical experts, to ones farther from the 
process, such as individual citizens, citizen groups, and 
youth. Many projects had multiple target audiences. They 
also varied in the degiec to which they actively courted 
audiences with divergent or competing perspectives on the 
issues. 

Project designs were similarly diverse. Three projects 
had events-oriented project designs emphasizing audience 
participation in a conference, roundtable discussion, or 
other carefully planned event Two projects had materials- 
oriented designs emphasizing the development and subse- 
quent dissemination of educational materials to identified 
audiences. Three projects had two-phase designs involving 
an extensive process of materials development followed by 



(Some groups were not included in ll o coalition, but those 
groups] couldn't have agreed that the sun had risen today. 

-coalition member 

One of the greatest sources of inertia is seeing [a public! 
issue but feeling alone and having no mint motion about it 

—project leader 
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use of the materials in a planned set of educational activi- 
ties. Other designs included a media-oriented project and 
two projects with emergent, locally based designs. 

Project events included seminars, conferences, training 
worl^hops, roundtabie discussions, and local meetings to 
view satellite or television programs and engage in discus- 
sion. Materials included booklets, educational kits or 
modules, curricula, video documentaries, a special issue of 
a journal, background papers, newsletters, conference 
proceedings, and contributions to a book. Other compo- 
nents of project design included survey* to identify issues 
or communicate opinion to policymakers, an educational 
tour, technical assistance, pilot projects, mini-grants to 
encourage local applications, study groups for deeper 
investigation of issues by smaller groups of individuals, 
broadcast of programs by satellite or commercial television, 
and requirements or expectations that learners teach some 
of what they learned to others. 

We found encouraging the evidence of ability to address 
a wide variety of issues. The necessity that public policy 
education be timely and relevant with respect to the 
ongoing political process meant that educators, to be 
effective, often had to accept other people s issues or 
definitions of the issues. The ability to address a wide 
variety of audiences was also important. The audiences that 
needed the most help were likely to differ with different 
issues, and different audiences were likely to need different 
kinds of help. Clear delineation of and familiarity with 
target audiences were essential for effective public policy 
education. Moreover, multiple audiences — or at least the 
ability to reflect multiple viewpoints on an issue — were also 
highly desirable. Otherwise, educators risked "preaching to 
the choir" and perpetuating the difficulty of moving Worn 
self-interest to public decision. 

f inally, the ability to draw on a "toolbox" of project 
designs was also valuable, given the likelihood that different 
audiences and different issues needed to be addressed in 
different ways if educational programs were to succeed. 
Although we were not always convinced that choice of 
project designs was well thought out, the diversity ol 
designs and design elements ac tually used provided a useful 
array of examples for future projects to borrow, adapt, or 
build upon. 



Vignette: Developing Public Policy 
Education [Programs on 
Unfamiliar Issues 



The Iowa Public Policy Education Project (PPEP) selected 
issues using an essentially democratic process and 
consequently faced the frequent challenge of addressing ■ 
issues on which project staff, and extension as a whole, 
had little experience or expertise. 

For its initial selection of issues, the project surveyed 
state legislators, statewide interest groups, and extension 
councils, asking respondents to rank a list of thirty-one 
issues by priority. A broad statewide cross-section of 
legislators, lobbyists, and community leaders was then 
invited to an Iowa Agenda Conference" at which the 
survey results were reviewed to develop an agenda for 
the project. Nearly 450 individuals participated in the 
conference. The final selection of issues was made by 
the project's coalition, the PPEP Council, an eighteen- 
member policy-making body. The council consisted of 
seven members appointed by the Iowa State University 
president, seven elected by participants in the agenda 
conference, and two each representing the state legisla- 
ture and extension field staff. 

The council decided that the project should cover the 
farm bill for its first program-a familiar issue to get their 
feet wet-and drug abuse for the second program. 
Mounting a credible and successful program on drug 
abuse was considered a big but welcome challenge. 
Topics for the second year were waste management and 
health care, and for the third year education and the 
state's budget crisis. 

For each topic, the project held a statewide satellite 
meeting in which 2,000 to 3,000 participants viewed a 
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program and engaged in local discussions at downlink 
sites throughout the state. In the project's third year, the 
programs were broadcast on commercial television. 

Project staff developed for each topic a videotaped 
documentary and a live statewide panel for the satellite 
town meetings. Because of the staff's lack of expertise in 
most of the topics, a campus/field staff task force- 
different for each topic-was created to draft an initial 
program outline and identify potential interviewees for 
the documentary and relevant interest groups to be 
invited to av information luncheon. A book of registered 
lobbyists and interest groups was maintained to help 
identify persons to invite to the luncheons. By the end of 
the project, the participants in these luncheons were 
considered Issue-oriented planning coalitions." They 
provided program content input, identified additional 
interviewees for the documentary, nominated focus 
group members, and helped promote the program. 

Whenever someone suggested that project staff were 
not kn wledgeable enough to be involved in a particular 
issue, staff would say, 'You're right! That's why we need 
your help!" 

Expertise for the documentaries was provided by the 
persons interviewed. To prepare for the interviews, staff 
spent more than two weeks gathering background infor- 
mation, emphasizing "researching the person" instead of 
trying to become substantive experts. The satellite town 
meeting programs were then rounded out by creating a 
politically balanced live panel. Local downlink hosts were 
responsible for the local panels and discussions. 



One element missing from many of the projects' interven- 
tions was "intensity/' Successful public policy education 
programs require educational interventions that are 
sufficiently intense or powerful to accomplish the in- 
tended aims for the target audience and issue. 

Most project designs resembled more a collection of 
events and materials and less a reasoned sequence of 
engagements for the same audiences, cohesively directed 
toward specified aims. Few projects undertook interven- 
tions with much intensity, even when audiences included 
ordinary' citizens, issues were acknowledged to be complex 
and multifaceted, or project objectives included increased 
participation in the policy process. Yet it seemed clear that 
less engaged audiences, more complex issues, and more 
ambitious objectives would require more intense 
interventions. 

For example, a one-day conference might have been 
sufficient to impart information to a single audience already 
knowledgeable about a public policy issue or about the 
policy-making process. But the same intervention might not 
impart sufficient information or promote dialogue among 
multiple audiences. And a one-day conference was unlikely 
to catalyze policy action among previously uninvolved 
citizens. 

Staff and coalition members in several projects acknowl- 
edged the limited contact most participants had with the 
projects or realized that some audiences were disappointed 
that the projects did not go beyond the provision of 
information or an opportunity to talk. In some cases, efforts 
were made, beyond the originally planned events, to permit 
or encourage additional discussion or local follow-up 
activity. Often, however, even when such needs were 
recognized, it was not possible to meet them with available 
resources. 

In our view, public policy education programs have the 
potential to help reverse the trends toward increasing 
disengagement from the policy arena in our society. These 
trends are powerful, so that effective challenges to them 
must be even more powerful. That calls for multiple versus 
single engagements, ongoing or sustained versus one-shot 
experiences, and educational philosophies that require 
active versus passive learning. 



There must be a phase in public policy education where 
people walk away with lewer answers than they walked in 
with. 

-coalition member 
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COALITIONS 



The W. K. Kellogg Foundation's public policy education 
project cluster was funded with the requirement that all 
projects be conducted by two or more organizations 
working collaboratively. This requirement addressed the 
inherent contentiousness of public policy debates in a 
democratic society. If two or more organizations that had 
different perspectives on a given public policy issue joined 
in creating and implementing an educational program, 
meaningful progress on this issue in the public arena would 
be more likely. What did the cluster evaluation team learn 
about the nature and role of coalitions in public policy 
education? 

LESSON 3 

The likelihood of success in public policy education 
programs is enhanced when programs are planned and 
implemented by a coalition of organizations. 

When we inquired about the benefits and drawbacks of 
working in coalitions, many of the benefits cited related to 
specific individuals or organizations (see Lesson 7). Others, 
however, related to an improved capacity to carry out 
effective public policy education programs. Examples 
included conducting a project that a single organization 
could not do alone, producing better educational materials 
as a result of interaction, access to more diverse audiences, 
and increased credibility for the project as a result of 
multiple sponsorship. In one project. Cooperative Exten- 
sion and the League of Women Voters collaborated; the 
league gave extension access to environmentalists and other 
"urban" audiences, whereas extension helped give the 
league entry into and credibility in the agriculture commu- 
nity. Almost everyone we interviewed acknowledged that 
their project could not have been done without the other 
members of the coalition or that it would have been a very 
different project had one organization done it alone. 

Each of the following results, effects, or consequences, 
clearly attributable to the existence of coalitions, was 
documented in at least one project: 



• reaching an expanded and more diverse audience 

• experimenting with a new educational format such as 
roundtablcs 

• working across disciplinary lines in creating materials 

• developing more balanced materials with respect to the 
policy alternatives presented 

• incorporating both content and process concerns in 
materials and events. 

• bringing key players together in discussions of policy 
issues (such as groundwater), the resolution of which 
required the involvement of multiple players (e.g., local 
and state government officials) 

• enhancing the credibility of the project via multiple, 
joint sponsorship 

• enabling greater risk taking in the project (because risks 
arc shared among all coalition members) 

• catalyzing action rather than just facilitating intellectual 
discussion 

In our view, several components of effective public 
policy education can be best fulfilled when two or more 
organizations collaborate. These components include (I) 
reflecting multiple perspectives on the issues at hand, (2) 
ensuring balance or fairness in the treatment of each 
perspective, (3) offering process assistance as well as 
information, (4) reaching multiple audiences, and (5) 
addressing issues that others have a share in selecting or 
defining. Although a single organization could execute 
these steps, it would probably not be done very effectively. 
Additional organizations can more easily reflect diverse 
perspectives; ensure fairness through their ability to 
understand and speak for different elements of the political 
universe; provide needed expertise on content or process; 
contribute knowledge of, access to, or acceptability with 
different audiences; and offer the expertise necessary to 
address different issties or definitions of an issue. 



Our coalition represents many diverse interests. We had to 
have a peace treaty just to get started. 

-coalition member 



ERIC 



10 

j 



LESSON 4 

There are many legitimate ways to form a coalition. 

The coalitions lor the eleven projects differed in size, 
scope, lypc of members, and structure. Size varied Irom tw o 
single-organization projects that used grass-roots strategies 
to try to develop community-based coalitions; to three 
projects with nine to ten organizational members; and one 
with eighteen individuals as coalition members. Five 
projects had coalitions with members from a single state; 
three had multislate coalitions: and three had coalitions 
most of whose members were national organizations based 
in Washington. D.C Six coalitions had both extension and 
nonextension organizational members; three contained 
only nonextension members; and two were made up of 
only extension members. The nonextension members 
included educational, advocacy, academic, and professional 
organizations; government agencies; nonextension univer- 
sity units; and individual leaders and citizens. 

Individual coalition members varied in the degree to 
which they represented themselves or their home organiza- 
tions and, when they did represent an organization, in the 
degree to which the organization as well as the individual 
was committed to the project. In the majority of projects, at 
least some of the organizational or individual coalition 
members had worked together previouslv. Most projects 
underwent changes in membership, often by individual 
representatives ol organizations in the coalition, but 
sometimes by the organizations themselves. 

Structure 

Two projects were characterized as developing coalitions. 
Each was comprised of a single organization working 
toward the formation of a coalition. Six projects had 
asymmetrical coalitions, consisting of one or more domi- 
nant organizations (often extension) providing leadership 
and stafl for the project and other organizations in support- 
ing roles. Three of these projects had a single dominant 
organization; the others bad two or three dominant organi- 
zations with additional organizations in secondary, sup- 



porting roles. Three projects had symmetrical coalitions in 
which all members collaborated more or less equally. Two 
ol these had two-organization coalitions in which each 
member provided staff for the project; the other had a ten- 
member coalition with a separate staff accountable to the 
entire coalition.' 

Definition 

The working definition and significance of the coalition 
concept also varied across projects. In a few cases, the 
supporting members of a coalition played a role that did 
not differ greatly from that of a simple advisor) 1 committee. 
In many projects, the coalition functioned primarily to 
enhance and pool resources between essentially similar 
organizations or ones that had worked together in the past. 
In other projects, the coalition developed and adopted a 
new agenda that reflected at least some interests of all 
coalition members. The latter coalitions were more likely to 
involve partners with little or no previous history of 
working together. One project developed its agenda 
beginning with the "arduous hammering out, word by 
word" of a common statement of principles with which all 
coalition members could agree. These principles served to 
guide and frame project direction and. in turn, all project 
materials and activities. 

Coalitions should help a project to address a full range of 
perspectives on an issue and to tap the necessary diversity 
of information sources. Such a mission can be fulfilled with 
a variety of coalition designs. Nevertheless, the next four 
lessons offer guidance on the types of coalitions that were 
more likely to serve this function. 

LESSON 5 

Strong coalitions have organizational partners distin- 
guished by several key characteristics. 

The coalitions varied in terms of a multidimensional 
quality that we called coalition strength. W e assumed that a 
strong coalition was one that would broaden and further 
the policy debate on an issue. Given this definition, four 



Ihe roahtion hes been (dntdstk. foi extension. . . \\v hovrn't 
dlvvavs (oopckitod w:'.hin the university system, let alone with 

t'Xtclllfjl dglvniK'S 

-extension administrator 
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projects had strong coalitions; tw o had medium-strong 
coalitions; three were "not really coalitions" (primarily 
because only one organization had decision-making 
authority for the project); and two projects attempted to 
develop coalitions as an intentional part of their design. 
What follows are the multiple dimensions of coalition 
strength. 

• Organizational partners in strong coalitions had 
discernibly different but not irreconcilable perspec- 
tives on the issues, from which they forged a com- 
mon project agenda or purpose. Among the eleven 
projects, the reasons for developing a particular coali- 
tion, beyond satisfying a condition of funding, were 
generally not clear. Coalition formation did not appear 
to be based on a carefully thought-out rationale, but the 
creation of a common agenda appeared critical to 
making a group a coalition. 

It was not difficult to develop a common purpose 
when the participating organizations had similar per- 
spectives. The more serious challenge lay in finding 
common ground between organizations that had 
differing but not irredeemably conflicting perspectives, 
That is. a strong coalition was not just a group with a 
common agenda but a diverse group with a common 
agenda — but not too diverse lest the group become 
immobilized by its differences. Among die eleven 
projects, two faced and met this challenge. One project 
with a ten-member coalition hammered out a common 
statement of principles (e.g.. "small- and medium-sized 
l .S. farms are important and should be protected") with 
which all members agreed and around which the 
subsequent educational program was developed. The 
other project was a coalition between Cooperative 
l:\tension. with its acknowledged rural and agricultural 
bias, and the League of Women Voters, an organization 
with a predominantly urban and suburban membership 
considered strongly environmentalist on agriculture- 
environment issues. 

Ai least two other projects held difficult and some- 
times painlul discussions in search ol "common ground. 
Rut in these cases the principal differences were among 
factions w ithin a single organization — namely, exten- 



sion. As such, these discussions were constrained at the 
outset by the boundaries of Cooperative Extension. 
Although such discussions may have facilitated changes 
within extension, we considered organizational change a 
side benefit, not a primary objective, of coalition-based 
public policy education (see Lesson 7). 

• Organizational members in strong coalitions brought 
complementary expertise or resources to the project. 

Organizational partners in nearly all of the strong 
coalitions brought complementary, project-related 
expertise or resources to their joint endeavors. Examples 
in two-member coalitions included process expertise and 
content expertise, national connections and local citizen 
networks, and access to the press and to substantive 
experts. That complementarity appeared to underlie a 
tendency in the projects for the different coalition 
partners to take on different project responsibilities — 
that is. for the partners to work on parallel tracks rather 
than on merged or more interactive tracts. But all kinds 
of tracks appeared capable of leading to successful 
educational programs; one route was not necessarily 
superior to the others. 

• Organizational members in strong coalitions had 
equal or almost equal project decision-making 
responsibility and authority. Partners in all the strong 
coalitions shared decision-making responsibility and 
authority equally or almost equally. No other projects 
had comparable symmetry in coalition structure. 

Parity in organizational structure and decision 
making was important in public policy education 
coalitions because it ensured that the common agenda 
reflected equitably the perspectives represented. Second, 
it enabled the coalition to model the democratic values 
ol equity, respect, and caring likely to be promoted in its 
educational program. 

Yet the experiences of a medium-strong coalition 
raised an important challenge to this argument. Power 
imbalances characterized organizational relationships 
within that asymmetrical coalition. Yet no one appeared 
troubled by such imbalances nor did the imba- Alices 
appear to impede project activities, which tended to be 



I'm convinced the more weird the Uvim is, the more? man- 
agement problems you have and the better the proiect is. 

—project leader 
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planned and conducted for each organizational member 
separately. One coalition member wondered if the 
obviousness of these inequalities allowed the partners to 
minimize professional rivalries. 

Organizational members in strong coalitions demon- 
strated organizational support for or commitment to 
the coalitions' projects. The importance of this crite- 
rion of organizational support was more obvious when it 
was absent. In other words, having strong organizational 
commitment did not typically contribute anything 
concrete or tangible to die project, but the absence of 
such commitment often seriously derailed or blocked 
intended project activities or directions. 

For example, in one otherwise strong coalition, lack 
of organizational commitment to the project among 
some coalition partners engendered serious challenges 
to, and subsequent redirections of, the project's dissemi- 
nation plans. In another otherwise strong coalition, lack 
of organizational support was painfully manifested by 
limited project awareness among many organization 
members and, hence, their limited attendance at project 
events. And, in a coalition comprised of elements within 
a single organization ("not really a coalition"), project 
staff expended enormous energy and time battling lor a 
toehold within the larger organization. In this case, even 
a modicum of organizational support, or simply the 
absence of active resistance, would have freed staff 
energies considerably to pursue the project objectives. 

• Organizational members in strong coalitions had 
individual coalition representatives with commit- 
ment to the project. In addition to the organizational 
commitment, the individual representatives to the 
project coalition also needed to be committed to the 
project. Such individual commitment was a characteris- 
tic of nearly all coalitions in this cliibtcr. 



LESSON 6 



Strong coalitions are not automatically formed when two 
or more organizations come together, even when such 
organizations meet the criteria specified in the previous 
lesson. Rather, strong coalitions have to be created and 
carefully nurtured through processes that are often 
laborious, painful, and full of compromise. 

Most of the strong, successful coalitions were character- 
ized by good working relationships among their members. 
Several projects had to rebuild relationships when changes 
occurred in coalition membership. Two projects success- 
fully resolved early conflicts among coalition members. 



Vignette: Exemplary Organizational 
Support 



The director of extension at this land-grant univer- 
sity strongly supported the public policy education 
project on water issues that was recently initiated 
by one of his faculty members. He noted that water 
issues were not only a national extension priority 
but currently a state priority for extension program- 
ming and research as well It seemed almost as if 
this extension director perceived the water project 
as a feather in his own cap. Such strong support- 
while perhaps not translatable into concrete 
resources or assistance-surely could not harm the 
progress and potential influence of the water 
project. 



Some of us love pumie Lind, diuJ some of us plow it up. 

—participant in a roundtable discussion 
involving farm and nonfarm interests 

An enviionmentohsl is ii humer whose well went IxrJ. 

-speaker at a workshop sponsored 
by one of the projects 



1 . ould not have done t'-r pinject if I were not : luli ptuk" -.of 
who couldn't one le^ - u*t departmental ^.:«oen for 
example]. 

-project leader 
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Coalitions arc ultimately comprised of individual people 
representing their organizations. "People are the key/' said 
one coalition member. In strong coalitions, the interrela- 
tionships among these individuals were characterized by 
trust, mutual respect, and rapport. And these positive 
interrelationships were carefully wrought and sustained by 
coalition leaders or staff skilled in group process and 
relationship-building. Members in two strong coalitions 
had worked together previously and so brought trust and 
rapport to the present project. In the other two projects 
with strong coalitions, coalition members were initially 
strangers and developed their positive working relation- 
ships during the project. In all four strong coalitions, 
members shared project authority and responsibilitv 
equally, were supported in their project activities by their 
home organizations, and, perhaps most imporlamlv. 
allocated time and energy to coalition-building activities. 

The experiences of one of the medium-strong coalitions 
perhaps most dramatically underscored the importance of 
strong personal relationships. In this three-member coali- 
tion, organizational commitment was strong, but there was 
considerable turnover in organizational representatives to 
the coalition, especially at the outset of the project. In the 
words of the project director, "The 'glue that makes them 
(coalitions) stick* together is in large part the rapport and 
trust which accumulates as people representing the organi- 
zations get to know one another and develop working 
relationships. Organizational change presents a challenge to 
the viability of coalitions. When individuals representing 
participating organizations leave, coalition relationships 
become less stable and I he project loses momentum. 
Members must make new investments to restore the 
coalition and bring a project back to its former productivity 
leveP* (Abdalla 1990. pp. 1 27-28). 

In short, coalitions needed to be nurtured. Too much 
attention to process could divert energies from substance or 
task, but cooperation would not necessarily be automatic 
when two or more organizations were brought together. 
Careful attention to "up-front" work at the beginning of a 
collaboration was required in most cases for a strong 
coalition and a succcsslul project. 



Vignette: A Staff Member's Vision 
Expands 

Three years ago, Arthur was a state-level extension 
specialist in natural resources. He spent half his 
time at the land grant university and the other half 
traveling around the state offering guidance to 
county extension staff. His work focused on the 
technical and educational dimensions of natural 
resource issues, and he interacted largely with 
other educators. 

Then, Arthur became the key staff member for 
his state in a public policy education project related 
to natural resources. His work on this project 
engaged him directly in the policy arena— both in 
national legislation and, more dramatically, in local 
politics and decision making. Arthur interacted not 
only with educators but with policymakers and 
directly with concerned citizens. His work now 
addressed not just the technical and educational 
dimensions of natural resource issues, but their 
social, policy, and action dimensions as well. 

In the process, Arthur became a ieader and a 
professional with a new direction for his career. No 
longer would the cloistered halls of the academy 
be sufficiently engaging and rewarding for Arthur. 
With his newly honed skills and experience, Arthur 
can be found out among the local folks, working on 
natural resource policies at the level where they 
directly hurt or soothe-still an educator, but no 
longer from a distance. 



At the end of this project, I will loci that I have a community. 

—coalition member 

In comedy (Odlitton, and sex. context ( ind timing <ire uitually 
important. 

-coalition member 



I his project could never hove happened in how*\ it vvoukr t 
have had a snowball's chance in hoi!, 
-extension specialist who worked with one of the projects 
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LESSON 8 



Coalitions typically benefit individuals and organizational 
members. But individual leadership development or 
organizational change should not substitute for meaning- 
ful progress in the policy arena. 

Among ihc individual professional development benefits 
cited by coalition members were exercising leadership, 
learning to listen and work with diverse participants in 
public policy sellings, and developing new knowledge 
about or sensitivity to unfamiliar issues, perspectives, or 
organizations. Examples included: gained] a better 
understanding of what is involved in trying to educate a 
group or the public in general and methods to do both"; "1 
learned about policy issues i didn't know much about *; and 
**[1 developed] an appreciation of diverse points of view." 

Coalition participants also cited benefits related to 
networking, including the opportunity to meet new people, 
work effectively and creatively with others, and make 
connections with people who have similar interests. 

Coalitions most often benefited organizations by giving 
them an enhanced image or visibility or by exerting 
pressure for internal organizational change. The latter was 
most evident within extension. The presence of other 
organizations in coalitions with extension, together with 
the requirements of the grant, helped extension (in some 
slates) lake on new issues, address new audiences, develop 
new perspectives and sensitivity, alter its image, and gain 
experience in "issues programming." Other organizations 
were similarly affected, as illustrated by the League of 
Women Voters new involvement in agricultural issues. 

Enhancing the ability of individuals to provide vision 
and leadership and stimulating ponderous organizations 
like extension toward responsive change were valued 
outcomes of coalition-run public policy education projects. 
But such outcomes alone are neither sufficient justification 
for requiring coalitions in public policy education projects 
nor sufficient evidence of project success. Coalition and 
project success should be judged instead on the nature and 
extent ol action occurring among program participants in 
the policy arena. Successful public policy education 
projects catalyzed such action, independent of their effects 
on the host organizations or individual project leaders. 



Public policy education can be effective in the absence of 
a formal coalition, but not in the absence of the spirit or 
broad intentions of a coalition — specifically, the commit- 
ment to meaningfully incorporating diversity by offering 
policy alternatives that reflect different points of view, 
and, at root, different values, in the form and function of 
the program offered. 

Members of one medium-strong coalition repeatedly 
challenged the ongoing attention to coalitions and coali- 
tion-building in the cluster. They wondered about the 
connections between coalitions and effective public policy 
education and raised the following specific challenges: 

• Public policy education inherently deals with diversity, 
even conflict, whereas coalitions need a common agenda 

• An effective coalition is fluid and bound by a common 
purpose rather than by a formal structure 

• Coalitions are an inevitable part of the public polic) 
process, so they do not need or warrant special attention 

Members of a stronger coalition believed that the formal 
coalition structure was much less important than the 
commitment to seek diversity, be inclusive, and offer dis- 
tinct perspectives. In fact, the formal structure might well 
interfere with or detract attention from such a commitment. 

Within the cluster were numerous instances of coalition 
spirit without a formal structure. Several projects, for 
example, ensured multiple perspectives, even when they 
were it represented on the coalition, by using ad hoc 
committees with diverse memberships to plan conferences 
or other educational events. Another project ensured 
balance in its educational materials by relying on advisory 
committees whose members criticized drafts from both 
traditional and more progressive viewpoints on food and 
agricultural issues. At least two projects used surveys to 
identify issues to be addressed, whereas another relied on 
brainstorming meetings with diverse groups of knowledge- 
able individuals. These were all "coalition-like" mecha- 
nisms but were implemented outside the project's formal 
coalition. 

Are formal coalitions, then, required for effective public 
policy education programs? Our answer was no. We 
concluded, how ever, that the essence of coalitions — the 
commitment to diversity and inclusiveness in perspective 
and values — was critical for meaningful public policy 
education programs. Without such commitment, public 
policy education programs could approach propaganda. 



15 1 f: 



REFLECTIONS ON PUBLIC POLICY 

EDUCATION 

A CASE STUDY 



"Thai was much less painful ihan I had anticipated." 
thought MariKn after the cluster evaluation teams first site 
visit to her public policy education project. Marilyn was 
tidying up her office at the League of Women Voters 
Education Fund, a long-standing, highly respected national 
organization with an educational public policy mission. 
" The chance to rolled on our first year of work, together 
wilh the key project people from our partner organization. 
Public Voice for l : ood and Health Policy, was not only 
useful; it was almost enjoyable." 

"And, especially upon reflection. I think our first year 
was quite successful. As 1 lold the cluster c valuators 
yesterday, one ol our important suece :s is the strong 
working relationship established between the two partners 
involved in this project — on both personal and organiza- 
tional levels. Public Voice is really quite something. It's 
such a small organization, and yet they are constant l\ in the 
news. I think that's because their director. Alison (irinswall. 
is masterful in working with the media. She's like a concert 
harpist, knowing exactly how to pluck reporters' "hot-story' 
strings, in fact. Public Voice exists largck to research and 
publicize major news stories about our food suppk . stories 
that intentionally disrupt the status quo And their exper- 
tise with the media really complements our own strong 
nationwide network ol community-based citizen groups 
and our tradition ol expertise in education at the commu- 
nity le\ol. So lar, this project has needed both." 

Specifically, during this project. Public Voices media 
expertise was tapped to publicize project events such as the 
annual food lorum. 1 hat fomm promoted dialogue on 
selected lood and larm issues among high-level 
policymakers with div erse v iewpoints — such as environ- 
mentalists and chemical industry representatives. Public 
Voices connections with the policy community in Wash- 
ington. H.C '.. were important in gelling participation in 
these forums. Press conferences, also organized b\ Public 
Voice, were held each year to announce the projects majoi 
publications. \nmulk . Public \ oice and the I eague ol 
Women \ oters each produced one publication oriented to 
its own organizational audiences. As important as anything 



else, the league gave credibility to this project as an educa- 
tional endeavor and long-standing expertise in working at 
the grass-roots level on public issues. The league lapped 
into iis local citizen networks for attendees at the trainings, 
which followed the food forums each year, and for annual 
recipients of small, local grams from the project. Both the 
training and the small grams were designed lo promote 
education and action about project issues at the state and 
local levels. 

Toward the end ol cluster evalualors' final site visit to 
the project. Marilyn reflected again. "The fact that Roger 
(who was the principal project staff person from Public 
Voice) and I got along so well, and that his economic 
bac kground in lood and farm issues w as a good match io 
my more substantive focus on natural resources and the 
en\ ironment. are important contributions lo this project's 
successful partnership. But. 1 think what really counts in 
the public policy arena is not so much a formal organiza- 
tional partnership, or coalition as Kellogg calls it — which 
can lake a lot of lime and energy lo sustain. Rather, what's 
most critical is the commitment lo respecting diversity and 
the openness lo the possibility ol compromise. Willi these, 
your work may actually meaningfully advance the public- 
policy debate. Without them, you'll jusi he tooling your 
ow n horn, and your message may well be losi amidst the 
cacophony of individual interests that is the public policy 
debate in ibis country today, rivaled only b\ a New York 
C'ily si reel embroiled in a traffic jam at quilting lime during 
a thunderstorm.'" 

A story about one of Public Voices publications illumi- 
nates an important issue in public policy education endeav - 
ors — the inherent tension between education and advocacv. 
a tension at the Ibrelroni ol this cluster of projects because 
ol Kellogg I'oundalion's strict prohibitions against using 
kinds lor advocacy purposes, (iiven iis tradition and its 
reputation, the league s publications were decidedly 
evenhanded and explicitly educational, presenting multiple 
points ol view on the issues. Public Voice, however, was 
more accustomed to working in an ad\ocac\ mode and 
lound itself with the challenge ol reorienting its work lo be 
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educational. In particular, Public Voices first project 
publication came to the attention of the Kellogg Foun- 
dation's project officer by way of several phone calls from 
irate agribusiness people in the Midwest. "How could the 
foundation responsibly sponsor this outrageous work!" was 
these individuals concern. While certainly not outrageous 
in most people s eyes, this publication did take a particular 
stand favoring strict controls over pesticides in farming, a 
stand not well received by many in the agrichemical 
community. Roger from Public Voice acknowledged thai 
learning to write something that did not explicitly take a 
stand took some time, but he — and others, including the 
Kellogg project officer — believed this was learned by the 
end of the project. 

At the same time, both Roger and Marilyn wondered if a 
strictly neutral stance on public issues was either possible 
or desirable. "It s hard to do effective public policy educa- 
tion without having an advocacy perspective that's rooted 
in close, ongoing linkages to policy issues and processes, 
and an advocacy base that can provide an outlet for people *s 
activism and passion," suggested both Roger and Marilyn at 
the end of their project. "A balance-bias tension in effective 
public policy education is inevitable and part of what 
makes it effective." 

At the end of their project, Roger and Marilyn also 
reflected on what it means to do public policy education, in 
particular, who is the most important public to reach? 
Here, they had created and implemented a successful 
project that involved various publics in education and 
conversation about timely food and farm issues such as 
pesticide use in agriculture. High-level policymakers, 
representing diverse viewpoints, came together for con- 
versation in the projects food forums. Interested citizen 
leaders from all over the country, especially representing 
the leagues established member-activists, participated in 
education and conversation with others about these issues. 
Like the recipients of the project's community grants, some 
workshop participants offered educational and dialogue 
opportunities of their own when they returned to their 
home towns. And thousands of requests had been received 
for the project's publications. 

Yet, despite these accomplishments. Marilyn and Roger 
were rethinking the meaning of public policy education as 
they neared completion of this project. Specifically, they 
believed that their work had reached the already-informed 
and involved but had not effectively capacitated local 
groups. In the wisdom of Tip O'Neill, all policy is ulti- 
match local policy. Thus public polic\ education at base 
should be local. In their next project, Marilyn and Roger 
would place more emphasis on local-level work and on 



energizing local people connected with leadership training 
and with opportunities to influence policy. This would 
enable the local projects to be supported longer and more 
intensively, thereby redirecting the focus of public policy 
education endeavors to those whose lives arc affected by 
the issues but who current!)' do not claim their voice in 
issue debates or decisions. 



IMPLEMENTATION 



As ihc cluster evaluation team observed and inquired about 
the projects implementation — challenges, successes, 
lessons, and adjustments — several themes emerged. One 
was the relative emphasis of different modes of education — 
prov ision of information, dialogue among persons involv ed 
in policy-making, and empowerment of people affected b\ 
the issues but not yet involved. Other themes included the 
role and importance of process assistance, relationships 
with the news media, and the line between education and 
advocacy. 

LESSON 9 

Different modes of public policy education are appropri- 
ate for different audiences, issues, and contexts. The 
projects found the dialogue mode to be a valuable alterna- 
tive or supplement to information provision. 

The projects in this cluster could be distinguished bv 
their primary mode of public policy education. We dis- 
cerned three different modes: 

Information provision, the most prevalent mode. was. 
at least initially, the single dominant mode 'of education in 
at least six projects. Forms of information included primed 
materials, conference presentations, panels, press confer- 
ences, and video documentaries. They contained lads 
about existing conditions and trends, causes of problems 
the positions and strategies of different groups, alternative 
solutions, and case studies of solutions that had worked in 
other settings. Although information about the policy 
process could have been included, in actual practice most 
information focused on the issues. 

In providing information, most projects endeavored to 
represent all viewpoints on an issue. Most also found that it 
is exceedingly difficult to provide fair and equitable infor- 
mation. One project underwent highly emotional battles 
between environmentalists and agriculturalists in curricu- 
lum development. Staff in another project reported literally 
counting the seconds ol exposure lor competing political 
viewpoints in a video program. In other projects, the 



development of educational materials took longer than 
expected, diminishing the role of those materials in project 
implementation. 

This mode of public policy education — and its concomi- 
tant reliance on materials — seemed insufficient for all 
except very elite audiences. High-level policymakers might 
benefit from a program intended to update their knowledge 
and understanding of a public issue. Those people already 
know how the system works and. more importantly, how 
to get their own voice and those of their constituents heard 
in the policy process. Most other audiences, however, are 
likely to need more than information to become meaning- 
fully engaged in the policy process. 

Dialogue, a second mode of education, w as dominant 
in no more than four projects, although many of the 
projects provided ai least some opportunity for dialogue. 
Whereas diverse perspectives on an issue could be clarified 
through information provision, the dialogue mode put 
stronger emphasis on creating a forum in which partici- 
pants from different sides of an issue could inform one 
another. Examples among the eleven projects included 
roundtable discussions of issues related to the farm bill, 
involving environmentalists, farmers, and others, and local 
discussion groups including representatives of different 
sectors of the food system. 

Several projects paid increasing attention to dialogue. 
One. which sponsored conferences on important regional 
issues such as health care, placed greater emphasis on 
convening diverse discussion groups and asking them to 
develop policy recommendations. Another, which initially 
invested considerable lime and energy in developing 
materials. later emphasized the process of bringing diverse 
interests together and relegated the materials to a support- 
ing role. 

Program participants required information about the 
issues. They aUo needed to know how other people were 
affected by the same issues, and required opportunities to 
develop the mutual understanding and agreement neces- 



I'his is beautiful stuff, but who asked foi it ; 

-potential user of materials developed 
by one of the projects 

I vvrythinR we've tried to do [in this pioicdj has horn twice 
.is hcird to do as exported. 

-project leader 
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saiy for gt iiiine issue resolution. Lack of agreement was 
arguably a bigger obstacle to the resolution of public issues 
than lack of knowledge. We are not suggesting a choice 
between information provision and dialogue. For many 
public policy audiences, both were needed. People involved 
in making public policy need to understand multiple 
perspectives, and this was often accomplished most 
effectively through a balance of information provision and 
opportunities for dialogue. 

LESSON 10 

The empowerment mode of public policy education was 
underused by the projects, suggesting a future need to 
reach out more concertedly to groups and individuals 
who are currently affected by but not involved in the 
policy process. 

Empowerment a third mode of education, was the 
least prevalent one among the eleven projects, and only 
three projects placed any significant emphasis on it. 
Education in this mode may have included information 
provision or dialogue but was distinguished from the other 
modes primarily by its target audience, people who were 
affected by public issues but wh^ perspectives were 
poorly represented in the policy- 'ting process. One 
project (featured in the vignette at right) attempted to 
mobilize citizens in a county with a large Spanish-speaking 
population; another was targeted toward youth; and a third 
had the explicit objective of reaching large numbers of 
citizens (rather than providing in-depth information to a 
smaller number). 

These rather limited experiences did not give us much 
basis for learning about education in the empowerment 
mode. But, consistent with the literature on empowerment, 
the projects with empowerment objectives were at least 
somewhat more likely than the others to (1) allow issues to 
emerge from the intended audience, (2) provide informa- 
tion about the policy process as well as the issues, and ( 3) 
include some form of special encouragement or assistance 
in taking action, especially at the individual level. Yet, most 
conceptualizations of empowerment maintain that some 
group, collective, or structural level of action or change is 



Vignette: Endeavoring to Empower 
Local Residents " • . « 

About twenty-five people had gathered in the 
spacious meeting room of the new county office 
building up on the hill. It was a pleasant Januaiy 
evening in this southwestern town, with a hint of 
dampness in the air despite the continuing drought. 
In fact, it was water issues that had brought these 
people to the meeting. Representing a host of 
varied constituencies-ranchers, city government, 
state parks, high school youth, sanitation agencies, 
and private citizens-these people had voluntarily 
gathered in this public forum to discuss their 
differing views on water issues. Like many areas in 
the Southwest, water issues in this town related to 
bdth the quantity and quality of water resources. In 
the local wisdom, "Around here, whiskey's for 
drinking and water's for fighting." 

The educators who had convened this meeting 
hoped that, via an open airing and discussion of 
contrasting views, ideas and priorities for a planned 
education program could be established. The 
selection of water issues as a top priority already 
represented diverse citizen input because it re- 
flected the results of a bilingual survey conducted 
in the area the previous fall. Now the task was to 
develop an educational program that would be 
responsive to citizens' needs and interests and that 
would engage people affected by water policy 
decisions but currently without a voice in these 
decisions. 



1 have not felt that it empowered people. I think people feel 
empowered when they see something happen. 

—participant, commenting on one of the projects 

They talk about one another rather than to one another. 

—participant describing the usual interaction 
between farm and nonfarm groups 
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also needed. Moreover, most of these projects lacked an 
effort to involve their audiences in a sustained set of 
activities over time. In other words, the intensity of inter- 
vention in these projects was not significantly different 
from the other projects. (In no project was the educational 
experience intense by design.) Yet much work on empow- 
erment argues that it requires a sustained commitment over 
time. 

Working with groups already involved in the policy 
process is a legitimate focus for public policy education, 
especially when the goal is to increase reciprocal under- 
standing among diverse groups. But such work might serve 
only to maintain the status quo with respect to who 
participates and has a voice if it is not accompanied by 
equivalent efforts to facilitate the participation of individu- 
als and groups who arc affected by issues but not yet 
involved. Finding ways to challenge present patterns of 
unequal representation and participation in policy-making 
should be an important goal of public policy education. 

LESSON 11 

Attention to process as well as content was a critical 
feature of the projects. Multiple types of process assis- 
tance are needed in public policy education endeavors so 
as to address the different process needs of the various 
audiences involved. 

One important form of process assistance in public 
policy education projects, addressed in Lesson 6, is facilita- 
tion of the internal processes vital to the internal function- 
ing of the project coalitions themselves. Strong coalitions 
needed to nurture their interpersonal and inter-institutional 
interactions and relationships. 

Relevant to the audiences of public policy education 
ventures, at least three otb r types of process assistance 
could shape such endeavors: first, paralleling process 
assistance for coalition members, there is training or 
leadership in the kind of group dynamics necessary to 
facilitate an effective policy dialogue. Such training or 
leadership would help project audiences to listen, under- 
stand, respect, and work with those holding other view- 
points on public policy issues. In addition, there is also the 
need to establish the conditions for such learning and 



I'm skeptical of process stuff, but I'm learning. I spent twenty- 
five years in the military where they don't teach a lot of 
process. 

-coalition member 



practice to take place; process assistance would provide the 
opportunities or forums for such dialogue. It can be argued 
that this form of assistance should undergird any other 
assistance provided in the policy education process. 

Many of the projects evidenced this form of process 
assistance. Their interventions ranged from providing 
explicit assistance in the form of in-scrvicc training work- 
shops for project audiences, to providing implicit assistance 
in the selection of a skilled group facilitator as project 
director, to simply providing the forums in which effective 
dialogue could happen. For example, one project provided 
process training for the leaders of their roundtable discus- 
sion groups, and another provided in-service training to 
extension staff as project implcmentors and audience 
trainers. Both of these projects included a printed leader- 
ship guide to accompany and extend the training. At the 
same time, however, several projects appeared to assume 
that process simply "happens" and requires no special 
efforts. In those cases, opportunities for group discussion 
sometimes went unfulfilled because of the absence of a 
specific process intervention. 

A second form of process assistance — rarely evidenced 
in the projects in this cluster — provides direct assistance to 
project audiences in understanding how the public policy 
process itself works and identifying opportunities for them 
to become effectively involved in the process at different 
levels of engagement. This assistance might vary greatly 
depending on the project audience. 

A third type of process assistance that several projects 
worked toward but did not often address directly, facilitates 
the translation of learning into action — providing assistance 
in knowing how to take effective action in support of a 
given position on a policy issue and providing leadership 
training for those intent on effecting change in the policy 
process. 

The nature and intensity of process assistance needed 
probably depends on the audiences addressed during the 
project, with relatively more emphasis on content for 
audiences already knowledgeable or experienced in the 
policy process and a more intensive process assistance 
strategy when a project attempts to reach a broader, more 
general audience. 
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Affirming die importance of process assistance in public 
policy education, several projects paid increasing attention 
lo dialogue as time went by. As many as eight projects 
tinned toward a greater emphasis on providing opportuni- 
ties for discussion or facilitating local activities among their 
audiences. Tw o projects created local funding opportunities 
during their final phases so as to bring local groups together 
lo discuss the issues and take action. 

In our view, this topic deserves greater attention in 
future projects. If effective public policy education demands 
moving beyond information prov ision to more active 
engagement in the policy process, then the projects them- 
selves should prov ide conditions for such engagement. 
Dialogue, respect for differences, consensus building, and 
search for common ground in forging viable public policy, 
recommendations are important elements that should be 
built into public policy education projects. Sinv.iy assum- 
ing that process will "happen" often leads lo d : sappoinling 
results or project impacts that fall short of w hat is possible. 

LESSON 12 

The news media are a potentially strategic resource for 
public policy education, but were underused by the 
projects. More attention needs to be given to the nature of 
the media's role and responsibility in public policy 
education. 

Only one project w as specifically media-oriented in 
design. Two oilier projects, how ever, developed sliong 
links lo the media. One capitalized on a coalition partners 
extensive experience and networks w ith the media; the 
oilier developed a working relationship with a network of 
commercial television stations. Many of the other projects 
reported adequate media covcragc'ol their events or 
publications. But these projects seemed lo view coverage 
primarily as publicity rather than a way of extending a 
project's educational message to a larger audience. 

The projects" composite experience with the media 
offered an opportunity lo explore the potential roles and 
responsibilities ol the media in greater depth. Projects 
engaged media personnel in several ways, journalists were 
invited lo attend press conferences, to observe public policv 



education events, and even to participate in educational 
events. 

"I he three projects based in Washington. D.C.. held press 
conferences to announce publications. Although some of 
those conferences sought to reach a broader audience than 
media personnel, one major audience consisted of national 
and regional journalists w ho might convey information 
about the publications and issues to their constituencies, 
W hereas press conferences offered die opportunity for 
information and dialogue with the publications" authors, 
the information eventually reported to the public could 
provide only basic exposure lo the issues presented. This 
was especially true because the media personnel who 
participated in press conferences w ere based in Washing- 
ton. D.C.. and it was difficult lo personalize public policy 
issues lor a grass- roots base when ihey w ere presented at 
such an abstract, national level. 

We suggest that more thought be given lo the relation- 
ship between the level of media (national, regional, local) 
and the stage ol development of a public policy issue 
(exposure, dialogue, action). Moving beyond simple 
exposure to issues might be most effectively handled at 
state and local levels where public policy issues can be 
made more concrete by examining specific manifestations 
ol a problem or concern. 

Some ol the projects with local public policy education 
events attempted lo invite the media to observe and report 
on those events. Because most ol the events lo which the 
media were invited were discussion-oriented, ibis approach 
enabled media personnel to report on policy issues at the 
grass-roots level by identifying and representing the views 
ol local actors on local policy issues. One project reported 
successful involvement of the media in two locally focused 
pilot projects, one of which included "news reports of the 
issues from a broader perspective" among its reported 
outcomes. Yet this approach was not enthusiastically 
embraced by all projects. One project actively discouraged 
inviting media personnel lo such events, (earing that their 
presence might provoke "grandstanding" by participants in 
a discussion-oriented event, precluding frank and open 
discussion. 



1 think there's tin impact when a laige group of people get the 
same information, in the same words, at the same time. . . .It 
makes the poopie in the hinterland feel like "I'm getting tin 1 
same message as [the state capital]." 

-project leader, commenting on a satellite 
telecommunications program 
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The major difficulty projects faced with the media, 
however, was attracting the interest and participation of 
media personnel. Local reporters were not always attracted 
by discussions of water quality or pesticide use. The 
definition of "newsworthy*' often seemed less focused on 
what people should know and more on what people would 
read. 

The projects* experience with the media led us to raise 
questions about the role and responsibility of the media in 
public policy education. Media personnel should be 
encouraged to move beyond simply covering "news** at the 
"sound bite" level to more in-depth reporting thai would 
further public policy education. This role may be more 
appropriate for interpretive reporters who write editorials 
and can explore issues in more detail. Public policy educa- 
tors seeking more in-depth media coverage might be more 
effective if they focused on engaging editorial writers who 
are willing to grapple with multiple perspectives on public 
policy issues in an in-depth manner. 

Two projects held educational events specifically 
designed to educate media personnel themselves. One 
series of events — press backgrounders — targeted young 
agriculture reporters and members of the "urban press" 
who did not necessarily have a "food and fiber' back- 
ground. These events sought to explain some of the 
complexities of farm policy. They were well received, even 
though there was not a direct link between a journalist's 
participation and the "copy" he needed to produce lor 
publication. A second project held a ■ communicators 
workshop*' for personnel in the religious media as well as 
lor educators within project coalition members ow n 
networks. Its goal was to examine specific public policy 
questions and encourage communicators to "disseminate 
information and promote education on agricultural trade 
within their networks.* 

Although the goals and strategies of these two events 
differed, some important similarities existed. Rather than 
inviting media personnel to report information, these 
events encouraged media personnel themselves to grapple 
with complex policy issues. Whereas the ultimate goal was 
to disseminate this inlormalion to their constituencies, the 
projects desired the press to have a greater understanding 



of the issues than could generally be obtained in their roles 
as reporters. 

The difficulty with these events was primarily logistic. 
Media personnel, under great pressure to file their stories, 
found it difficult to lake lime to obtain background infor- 
mation thai was not immediately convertible to print. Thus 
participation in the projects' media events required a 
supportive news agency that saw the long-term value of 
more educated jourralists. Nevertheless, educating the 
media on public policy issues might help minimize over- 
simplification in reporting b\ developing reporters' appre- 
ciation of the complexities ol public issues. 

Although the media might be a target audience for 
public policy education, they are also potential partners 
with educators. The need for such collaboration is evident 
in the shortcomings of either group working alone. The 
media are often criticized lor filling citizens with discon- 
nected facts while failing to help them make enough sense 
of public life to be able to participate effectively (Graber 
1^88; l.insky 1988). ['durational programs, designed in 
part to rectify such conditions, inevitably suffer from limits 
in the numbers of people reached. The vignette at right 
illustrates one possibility lor collaboration between public 
policy educators and the media. 

LESSON 13 

Tensions between education and advocacy are inevitable 
in public policy education, but established guidelines 
saying "educate, don't advocate" did not give the projects 
adequate guidance. 

All of the projects in the cluster endeavored to uphold the 
traditional public policy education model ol informed 
debate, representative discussion, and consideration of all 
policy alternatives and their consequences. Nonetheless, 
most struggled to find and maintain the line line between 
education and advocacy. In projects with advocacy organi- 
zations as coalition members, the education- advocacy 
tension was often overt. An early publication prepared by 
an advocacy organization in one project was criticized lor 
being biased, but subsequent publications from the sam< 
organization were written in wa\s that did not explicitly 

Sometimes I wondei if the younger ag reporters even have 
ftudens 

-participant in one of the projects, 
a veteran agriculture journalist 

Wi-'rr not <i < filnlysl loi t h.in/c, but <i < ntnlyst for ollur. to 
\>\\u<,] dlioijt ( I tdi ny 

-project staff 
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vignette: An Emerging Partnership 
with Commercial Television 



The Iowa State Public Policy Education Project (PPEP) 
made extensive use of satellite technology in deliver- 
ing its public policy education events. From the 
beginning of the project, participants gathered in small 
groups in their communities for statewide public 
policy education events in nearly 100 downlink sites. 
They viewed videotaped interviews with players in the 
policy process, such as congressional representatives 
and lobbyists, followed by a statewide panel discuss- 
ing the issues. Some events included a question-and- 
ar.^wer period with a local panel of experts. Partici- 
pants were then.able to engage in small-group 
discussions and to indicate their preferences on the 
issues discussed. 

PPEP linked up with commercial television during 
the third year of the project. The previous satellite 
programs had received favorable feedback, and 
people had asked project staff, "Why aren't you on 
public TV?" The staff approached public television, but 
the latter was unable to cooperate because of its own 
budget cuts. The PPEP staff then approached a 
commercial television station that already had a 
statewide network for sportscasts. To fund the broad- 
casts, advertising was sold to pay for air time, and 
Kellogg funding covered the production costs. 

Even before he was contacted, the station manager 
at the lead station in the network had already been 
"attracted to the quality of the programs" and "felt we 



could make money" by airing the programs in coop- 
eration with PPEP. He noted that commercial televi- 
sion would not have produced a program like PPEP's. 
"We would have set the governor down and the 
leader of the loyal opposition and let them yell t 
each other. But only viewers who were already 
convinced would watch." By using PPEP's approach, 
however, these programs attracted a relatively large 
audience. Whereas the. Nielsen ratings indicated 
65,000 watched one program, in contrast to the 
typical 200,000 for a prime-time network program, 
the ratings compare favorably with the viewership for 
public television and are much higher than the 2,000 
to 3,000 participants in previous satellite meetings. 

The PPEP programs benefit the station in at least 
two major ways. First, the public affairs programming 
enhances the station's image. It can be very expen- 
sive, however, for the stations to produce good public 
affairs programming themselves. Their partnership 
with PPEP enables them to air good-quality programs 
that move the station away from presenting politics as 
"thirty-second announcements." Second, the stations 
make money, which is more necessary because of 
declining national revenues for commercial television 
as a result of competition from cable. Stations see the 
need to air more local programs, which attracts local 
and statewide advertising. 



In policy education, controversy is not only inevitable but 
desirable sometimes. . . .This can be frightening. 

-project leader, on video about the project 

I've tried to present both sides, but they keep sniffing out my 
biases. . . .They're in the business of not offending anyone 
One can't take a stand at all; that is strictly forbidden. 

-materials developer for one of the 
projects, talking about extension 
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take a stand. All organizations in all projects endeavoicd 10 
adhere to the traditional public p' ,: 7 education model, ai 
least for the project at hand. It was generally recognized 
that participants in project activities — and, in other con- 
texts, coalition members themselves — could advocate as a 
result of what they learned, but that advocacy had no place 
in the projects themselves. 

Education-advocacy tensions were often more subtle in 
projects with exclusively or predominantly extension 
coalition members. Extension educators were generally 
familiar with the traditional public policy education model, 
so that education versus advocacy was openly contested in 
only one extension-dominated project. Yet, underlying 
several projects were important disagreements about what 
constituted neutrality on the food and agriculture issues at 
hand. Such conflicts were typically initiated by spokesper- 
sons for production agriculture (objecting, for example, to 
alleged environmental, nutrition, or sustainable agriculture 
biases), but they often resulted in noncxtension members of 
the coalitions or other outside observers maintaining that 
extension was not as unbiased on these issues as it claimed 
to be. One project whose dominant coalition member was 
extension struggled to hammer out curriculum materials 
acceptable to both environmental and agricultural interests. 
Materials for another project were repeatedly criticized by 
representatives of both traditional and more progressive 
viewpoints on food and agriculture issues, generating 
delays of over a year in materials development and pilot 
testing. These conflicts were often emotional, striking at the 
heart of basic assumptions about the ability of science to 
solve problems and provide the best answers. In most cases, 
the projects and especially their coalitions were viewed as 
steps in the right direction, helping to expand the range ol 
issues, alternatives, constituencies, and interests repre- 
sented in extension s educational programs. 

It seemed to us that these conflicts were not over 
education versus advocacy so much as over balance versus 
bias. The issue was not whether a range of alternatives or 
viewpoints was being presented (it was, in each case) or 
whether a particular outcome was being advocated (it was 
not). Instead, the question was which alternatives or 
viewpoints were being presented and which ones were left 



out. Similar questions could be raised about the projects of 
coalitions whose key members were advocacy organiza- 
tions. Project materials, though developed and presented in 
a neutral fashion, might selectively feature content leading 
most reasonable individuals to agree with a coalition's 
position on the issues. 

Another source of dissatisfaction with the traditional 
public policy education model was the concern that at least 
some audiences needed more than the ••neutral" presenta- 
tion of information before they could translate what they 
learned into action. Some projects were criticized by field 
staff or participants in the educational activities for having 
"big bang' educational programs and then not being 
around to help with follow-up. Follow-up was less a 
concern m projects with audiences that were relatively 
familiar with the policy process, such as policymakers, 
policy professionals, or the press. In fact, the lack of 
attention to follow-up may be a legacy from previous 
educational programs that had more sophisticated audi- 
ences. Some projects, such as one with youth as its audi- 
ence, encouraged or required participants to "do some- 
thing" as a result of their learning. Several projects gave 
increasing attention to follow-up as time went by. Some 
asked discussion participants to develop policy recommen- 
dations. Others provided seed money to stimulate or 
facilitate follow-up at the local level or at least talked about 
bringing key individuals togther to discuss the next steps. 

We believed that the "educate, don't advocate" guide- 
lines needed to be rethought. Emphasis on the neutral 
presentation of alternatives and consequences, and the 
corresponding ban on advocacy, worked for some projects, 
but others increasingly found such approaches inadequate. 
They oflered too little "so what 7 " for some audiences and. 
in other cases, failed to provide help when there were 
disagreements about what is neutral. 

LESSON 14 

Most projects agreed on a mode! of public policy educa- 
tion emphasizing what could be summarized as "balanced 
education plus follow-up," but the desirability of follow- 
ing that model is not yet settled. 



How can you be as effective as Meryl Streep without being an We don't have to answer the questions. We just have to figure 
advocate for one position? How can you advocate for in- out how to phrase the questions. 

formed choices'? -project staff 

—member of project advisory group 
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By the end of iliis round of projects, staff and other 
coalition memhers from a majority of the projects seemed 
to agree on sev eral points regarding education and advo- 
cacy. Some disagreement remained, much of it between 
extension educators and educators from advocacy organiza- 
tions. The following points, on which there appeared to he 
substantia! agreement, were heavily influenced b\ the 
extension educators, who constituted a majority of the slafl 
and coalition members across the elev en projects. 

• It is all right to advocate for more information in the 
policy process on the grounds that more information 
leads to better decisions. It may also be okay to 
advocate for more participation on the grounds that 
it, too, leads to better decisions. Advocating the latter, 
however, is riskier because policymakers or other 
influential participants in the policy process might not 
want more participation. Leaders of at least one project 
openly acknowledged that they avoided advocating 
increased participation even though they personally 
believed in it. 

• It is okay to advocate for attention to a particular 
topic and even for a particular definition of the topic 
as long as one does not advocate a particular out- 
come. For example, the leaders of one project said ihcv 
were advocates of attention to rural issues and of an 
approach to rural policy that was broader than most 
traditional definitions of rural development. "lo that 
extent, we've got an agenda."" they said. "But. when it 
comes to answering a question which we get frequcn th- 
at both the slate and federal level — AVhai do you guys 
think we should do?' — we're very straightforward in 
saying. That's up to you."' Agreement with this position 
w as not unanimous, however, and one project did 
relinquish the choice of w hich issues to address, turning 
it over to an essentially democratic process. Nonetheless, 
hardly anyone seriously questioned the practice of 
choosing m advance the issues to be addressed. As 
another project leader said, the project could have 
chosen different issues — ones of greater interest to urban 
people — but. then, w hat reason would there be lor her to 
be the project leader? 



Do we take neutral mdtcrwl and "baptize" it, so thrit some 
otgani/otions emphasize the faintly, others the poor? Could 
we get people to use the materials if part of the dynamic is 
t! ictt the advocacy positions various groups hold are iruide 
explicit? 

-coalition member, wrestling with the question of how to 
get project materials disseminated and used 
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• If educational events or materials do advocate 
particular outcomes, they should be balanced by 
equally articulate statements of alternative positions. 

This point was most often made in relation to eon fere nee 
agenda*, in which the value of provocative speakers w as 
sometimes recognized. It was generally acknowledged 
that perfect balance was rarely possible, but that there 
should be at least reasonable balance. One project leader 
described it as "treading evenly on even one's toes," A 
related rule, suggested by at least one coalition member, 
was that advocacy should not be manipulative or 
antagonistic ('"going beyond education"). 

• The preferred alternative to advocating particular 
outcomes was balanced education, but what docs 
that mean? When we examined our data for language to 
describe it, we found such phrases as "clarifying issues," 
"explaining the environment."' "outlining options," 
"describing innovative programs." and "asking ques- 
tions.*' This conception of education w as similar to the 
"alternatives and consequences" model familiar to many 
extension educators. But, for some at least, "balanced 
education" could also mean laying out a particular 
solution — and not necessarily a full range of options — as 
long as the educators served primarily to ask questions 
("Is this something you want lo consider?") and catalyze 
discussion. 

• There was widespread agreement that balanced 
education needed to be supplemented with some 
kind of follow-up assistance. Coalition members in 
one project noted that act ion -oriented people are often 
frustrated by programs that "stop at education." In the 
end. many pro j eels appeared lo agree on this model of 
"balanced education plus follow-up." although actual 
attention lo follow-up was delayed or remained limited 
in f.'weral projects. This prompted us lo w onder if fear of 
advocacy sometimes leads public policy educators to be 
less helpful to their audiences than they might be. 

Not everyone agreed with the model of "balanced 
education plus I ol low-up/" how ever. Disagreement 
occurred at two ends of a continuum. At least some staff 
or coalition members maintained that balanced educa- 



My problem is that it seemed self-indulgent and a luxury to do 
"pure" public policy education. 

—project leader, commenting on the idea 
of neutral, "objective" education 
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lion is sufficient without follow-up when a topic is very 
controversial so that interest in it is high or when experts 
anil lay people agree on the existence of a problem. At 
the other end of the continuum was the argument thai, 
even with follow-up, education without advocacy will 
not he effective. 

The latter argument was most common among 
advocacy organizations but was not limited to them. The 
staff of one project with advocacy organizations in its 
coalition maintained that nonadvoeacy not only was a 
frustration for advocacy organizations, but failed to serve 
the purposes of education itself. In their view, effective 
public policy education required advocacy so as to pro- 
\ide an '■outlet for peoples activism and passion.* Thc\ 
considered Kellogg s nonadvoeacy requirement unrealis- 
tic and unhelpful. Leaders of another project with 
advocacy organizations in its coalition said they were 
still "unsettled" about the education-advocacy issue. 
Most members of their coalition concluded that balanced 
education was a desirable way to make advocacy better 
informed or of forcing people to think. A nrnority of 
that coalition disagreed, however, doubting whether 
nonadvoeacy was sufficient to meet the needs of advo- 
cacy organizations. As one coalition member said. "A 
balanced group will get nothing done." Fvcn extension- 
dominated projects had some disagreement about 
whether advocacy — and the resulting controversy it can 
generate in an educational program — inhibited learning 
by being a distraction or promoted it by stimulating 
interest. 



We naturally learned more about education from this 
cluster of projects than we did about advocacy. We 
could see the limits as well as the value of "pure- 
education, but we lacked the same opportunity to test 
the value or limits of advocacy. Advocacy could certainly 
promote one-sided and poorly informed action, but 
presumably it could also promote solutions that result 
from a genuine effort to understand and reflect the full 
range of perspectives on an issue. W ould that be more or 
less desirable than education that was so balanced that it 
deflated people's enthusiasm and produced no action at 
all? It seemed to us that the case was not yet closed. 
Projects should have the flexibility to continue exploring 
variations from the model of education narrowly 
defined. 



iVnpIr <}< t hum emotion, not iu*'t limn the liict\. lint's why 
sc lenlisK nviko U-mhk' salesprople. 

-workshop speaker, a public official 

tiling.. 01 about (Mining ioi i.iivi *t things Oi maybe we've 
rarer U i n a: 1 rKtiviM and don'l know hn-.v to do il 

-project leader 
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THE DILEMMA OF BALANCE 



A CASE STUDY 



The Global Food Web (GFW) project began with a week- 
long youth conference in which leens focused on i lie 
relationship between human nutrition, food supply, and the 
environment. Rather than simply learning content, the 
young people were encouraged to find ways to become 
involved in these issues in their local communities. As one 
aspect of this involvement, teen teams were asked to "peer 
teach" the material to other students at home. 

We met Billy and Cathy after they had completed an 
hour-king program lor 300 sixth graders on the GFW. Billy 
began to explain what they did with the students. "We took 
the information and activ ities we learned at the GFW youth 
conference last summer, and taught that to the sixth grade 
classes. Nutrition, food supply, and the environment. . .in 
an hour! There is a lot of material to cov er, but it never 
occurred to us to leave any part out. since thc\ are so 
interconnected. We began by discussing basic nutrition and 
then involved teachers and other adults in a lap sit exercise 
illustrating the six basic nutrients and the consequences of 
removing some of them. We tried to help the students see 
that malnutrition is sometimes caused by not eating the 
right foods." 

C athy added. "This part might seem kind ol boring at 
lirst. but something happened at the conference last 
summer that showed us just how important it is to under- 
stand basic nutrition. This rock singer came and started 
trying to convince us to become vegetarians. We'd just 
learned about nutrition, and I didn't see how we could gel 
all the basic nutrients we need if we did. Wo talked about 
his views the rest ol the w eek! 1 lis perspectiv e dchmiclv 
made me think. 1 guess I can like his music without 
agreeing w ith him." 



"1 think students need more controv ersy in conferences like 
the Global Food Web." commented Joanne, a member ol 
the coalition lor the GFW project. Thc\ need to be able to 
debate issues, and learn to separate (act from opinion. This 
is where the best learning takes place. 1 low ever, when il 
i ame to the Gl W curriculum, while there was plentv ol 
controversy, within the project coalition — particularlv vv ith 
respect to the environmental curriculum's portiav.il oi 
agriculture — all ol it was removed bv the time the teens 



weie exposed to the issues. Think of what the teens might 
have learned if they had participated in that debate. . . ." 



Susan, the GFW project director, shook her head. "Boy. 
was I naive. I had no idea how strong the animosity was 
between environmentalists and agricuh .tralists. Let me tell 
you about that fall meeting where the project coalition met 
to finalize the curriculum for the GFW. The human 
nutrition and food supply curriculum, written by a team of 
extension specialists, had finally been approved alter being 
examined 'with a fine-tooth comb'. The environmental 
curriculum, primarily written by specialists outside of 
extension, was on the table. An extension specialist in 
production agriculture who had recently joined the coali- 
tion staled. Fni concerned that there is too much opinion 
here, and not enough fact — loo much mass media and 
emotion. You can't just talk about eliminating pesticides. 
The food supply side has to be considered. There needs to 
be balance in this curriculum. Fm not supporting this 
program if it's going to hurt agriculture/ 

"After much discussion, another member responded. 
W e have to face the fact that this curriculum is a product 
by committee. 1 think balance will be impossible to achieve. 
Fvcrv time a new person enters the equation the average 
changes and the balance mov es. I lb ink the best we can do 
is arrive at a compromise. " 



The reactions of an outside specialist who reviewed the 
GFW curriculum illustrate the tenuous nature ol "balance. " 
"The struggles with the GFW curriculum arc really ironic. 
In m\ opinion, the environment curriculum focuses on the 
issues where experts agree. The overall effect on students. 1 
would think, would be a clear understanding ol environ- 
mental issues. The food supply curriculum — which didn't 
engender neativ as much internal controversy — is less 
representative. The uirriuilum maintains that enough food 
can be produced. Distribution is the problem. But more 
production, both within the I nited states and abroad, is 
presented as the only solution. Other allernaliv es. such as 
population control, are studiously avoided. 
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"In my view, this is much less 'balanced' than the 
environment curriculum. Inn perhaps was less controver- 
sial because people with similar perspectives wrote and 
leviewed the material. In neither case wore the teens 
engaged in the controversy, hut lor different reasons, in the 
environment curriculum, the controversy was prcviouslv 
resolved, and in the food supply curriculum, the contro- 
versy was essentially ignored." 



Billy said. "\\ c always like learning about the environment, 
but what stood out lor me from the CFW conference is the 
food supply curriculum. The real problem with world 
hunger is a distribution problem. . .we have enough food. 
We had learned about nutrition and the environment in 
school, but I didn't know anything about \\xk\ supplv 
before the (il'W conference. W e did one of my favorite 
activities on food supply for the sixth graders this morning. 
We \iarl with, a couple of people in a square, and toss them 
some candy. Then we keep adding people to the square, but 
toss them the same amount of candy. It really illustrates 
that more food needs to get to countries with growing 
populations." 

Cathy added, "Learning about the food supply helped 
me understand more clearly how important il is to protect 
the environment. We did another exercise this mornino 

o 

with a melon that illustrates the small fraction of the earth 
suitable and used for food production and the tinier 
fraction ol fresh water. Agriculture and protecting the 
environment need to go hand in hand." 



Susan observed, ' W e eventually got the controv ersy out of 
the curriculum, but I think some of the life went with it. 
We obviously did not plan to have the rock star advocate 
vegetarianism, and about died while he was up there, but 
his remarks really energized the teens, and put back some 
ol the fire that was missing from the curriculum itself." 



Cathy explained. 'When we got home from the conference, 
we brainstormed all of the realistic possibilities lor wavs we 
could get invoked in the issues we had learned about. 
Teaching 1 he younger kids has been important, but we also 
wanted to get directly involved in the issues themselves." 

Billy said. "I thought getting involved would be difficult, 
until I realized that these kind of issues are right here, not 
only in other countries. For example, one ol the problems 
we have here in our count}' is a toxic waste dump that has 
been seeping into the w ater supply of one housing develop- 
ment. These people weren't getting any clean water at all. sc 
we collected drinking water for them. We also researched 
the problems with toxic waste dumps and got our facts 
straight, then went to a commissioners meeting. 

"I was so mad!'' Cathy interrupted. "The commissioners 
treated us like little Lids. 1 said. "What about mv future?' 
W hat about my kids?' I think their attitude changed a bit. 
but they were under other pressures. But w e keep checking 
in to see what they're doing about the problem. We're 
learning <hat we can have an impact on these issues." 



As we reflected on our conversations with these partici- 
pants in the CiFW. we noted that much ol the struggle at 
the project coalition level was focused on developing 
"balanced" curricula. This underscores the idea that 
teaching youth to think correctly about these issues is most 
important, even though the definitions of "correct think- 
ing" on any issue are never static. And. although the teens 
obviously gained insights from their experience with the 
content, they were more energized by learning that they 
could form their own judgments and act on this newlv 
acquired knowledge. We are left with the question, how 
might this project have been different if the conference had 
devoted more lime to teaching students how to think rather 
than what to think about these issues? 



'The traditional model for teaching is to present the truth." 
Joanne observed. "But when il comes to public policv 
issues. I think we need to leave it a little more open, 
leaching kids how to think, rather than what lo think. 
Teens learn quickly not to challenge 'the truth; and while 
they do learn, they are really molivaied when they have lo 
think on their own. However, it's more lisky lo leach ibis 
way. because we lose control of the outcomes. But the teens 
may initiate better outcomes than the «>nrs we caielulK 
orchest rate." 




IMPACTS 



Whai difference did ihe projects make? hi this chapter, we 
discuss lessons regarding the projects out conies. Data on 
outcomes were taken from the projects own internal 
evaluations, supplemented by interviews and observa lions 
by the cluster evaluation team. Because of the Kellogg and 
Farm Foundations understandable interest in the potential 
for continuity beyond the termination of Kellogg funding, 
this chapter also includes lessons on sustainability. We 
begin with a discussion of the projects evaluation strategies 
and some lessons in regard to that topic. 

LESSON 15 

The projects developed strategics for evaluating project 
implementation, coalition development, and project 
impacts. But more purposeful attention to evaluation 
design would be helpful at the beginning of project 
development. 

Although all projects conducted evaluation activities in 
compliance with grant requirements, the scope and depth 
ol these evaluations varied. On one end of the spectrum a 
project conducted an evaluation at the end of the project 
because it was required but viewed it as an annoyance. On 
the other end a project's evaluation team participated fully 
in the project throughout the grant period. 

The eleven projects evaluated different aspects of their 
activities, including events and materials, progress of the 
project as a whole, and coalition development. Some 
projects also focused on project impacts, particularly those 
on individual participants. Several methods were used to 
evaluate public policy education endeavors, including 
paper and pencil surveys, self-reflection, and journals. 

Many projects evaluated the implementation of their 
activities, both events and materials, by means of paper and 
pencil surveys at the completion of a project event or by 
reviews of educational materials. Most of these included 
participant ratings of speakers and activities as well as 
participants assessments of their increased knowledge of 
issues, changes in altitudes, and plans for follow-up action. 

To assess overall implementation strategics, some 
projects viewed self-reflection at the end of an event as an 



important mechanism for determining midcourse changes. 
This informal self-evaluation was conducted either by staff 
or by a committee appointed by the organizations in the 
projects coalition. Staff on two projects kept a project 
activity journal that documented implementation activities 
and decisions throughout the course of the project. This 
enabled project personnel to monitor progress as well as to 
have a record for retrospective review. 

Two projects developed innovative approaches for 
monitoring and evaluating coalition development. One of 
these is summarized in the following vignette. The other 
project conducted telephone interviews with coalition 
members focused on coalition building in the project. This 
project also conducted a final "endings and beginnings" 
conference at which coalition partners could report on 
project activities and share lessons they had learned. 

Projects also made efforts to track and document the 
impacts of their public policy education endeavors on 
individual participants. Few purposeful attempts were 
made to evaluate the impacts on the public policy issues, 
although this was recognized as important. 

Some projects defined exposure to issues as an outcome 
and kept track of the number of people who participated in 
or were reached by the project. One project documented 
the number of people participating in its downlink satellite 
conferences. In another project, county teams of teens were 
expected to "peer teach" material received at a conference 
to at least 1 50 others. After the first three ir.onths of the 
project, county team leaders reported that over 400,000 
other people had been reached in this manner. This 
number was established through documentation from the 
county team leaders describing their activities and atten- 
dance figures. Another project employed a clipping service 
to track news of the project in the media. 

To assess impacts on the participants themselves, self- 
report data were used. Some projects did follow-up phone 
interviews or surveys, and at least one project conducted a 
follow-up group interview with participants. The interviews 
and surveys asked whether project events affected them in 



After (hdt one meeting, they would h<we made us rat the 
forms. 

-project leader, on hearing that some projects were 
asking participants to fill out evaluation forms 
following each educational event 
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Vignette: TOP Evaluation Strategies 

The Trade and Development Project (TDP) hired two 
professional evaluators in March 1989. This evalua- 
tion team began by assisting the project in establish- 
ing a working coalition. They interviewed the mem- 
bers of the project's oversight committee as to their 
goals, objectives, and expectations with respect to the 
project. These goals were "rich and varied' 7 but fell 
into categories, which the oversight committee placed 
in priority order. After several revisions, the oversight 
committee arrived at a common set of goals. 

These goals were then operationalized into an 
"objectives-based evaluation plan/ 7 There were too 
many objectives for the evaluators to examine on their 
own, so the oversight committee agreed to work with 
the evaluators in identifying representative groups for 
field testing the materials as well as developing a 
process for documenting, tracking, and reporting 
results. 

This working arrangement between the oversight 
committee and the project evaluators became a 
critical dynamic in this project, which struggled with 
disseminating its materials. Because the evaluators 
were carefully monitoring the materials being sent out, 



they quickly identified and brought the dissemination 
problem to the attention of the coalition. As a result, 
although the evaluation team fulfilled the mo;e 
traditional roles of monitoring field tests, observing 
events, and suiveying participants, it also became 
involved in a rather nontraditional evaluation capacity. 
This team attended every oversight committee 
meeting and, in the initial stages, frequently found 
itself cast in the role of the "heavy," holding oversight 
committee members accountable for the dissemina- 
tion and evaluation-related work they had agreed to 
do. On occasion, the evaluation team also reminded 
the committee of their common goals and helped to 
keep the coalition on its agreed-upon course. 

The evaluation team conducted a "midcourse" and 
final evaluation as well as ongoing monitoring of 
project mateiials and events. All of the evaluation 
efforts in this project were discussed by the oversight 
committee, and the findings were used to help the 
coalition make decisions. These evaluations helped 
the coalition redirect its energies and explore alterna- 
tive strategies for reaching its goals. 



ways such as increased attention to the issues, changes in 
view*, participants" use of information provide 1. and anv 
activities undertaken since participating in project events. 

Despite the attention given to collecting evaluative data, 
only one project articulated an underlying evaluation 
design. Because evaluation designs make different assump- 
tions about what is ■'true" and how those truths can he 
known, adopting such a design early on w ould have 
enabled projects to he clearer about what "counted" as data. 



including what constituted project outcomes. They would 
also have had a road map for planning the evaluation 
methods used. An evaluation design would have provided 
criteria lor assessing the quality of project findings. Com- 
plying with those criteria would have enabled project staff 
to be more confident of their evaluation results and would 
have given outside observers a standard for making their 
own judgments concerning the projects implementation 
and outcomes. 



Ihv .non*hii',sUMWi£ ( nu h.s ';>!;;' uppiodch lo ovdkwtius] 
CKMU'd -.; positive (tftitudr rtnniig piO|c(1 \h\\h>. :p jo's 1n.\-}'<l 
tho project )\ an e\puinviit n pubU puli^ utk.<ali":i 

—project leader, in final report 



W)U Jmol (i 1 wIvitCAvi flutters ond ctoim cdtdtevof kills, 
-project leader, asked how he can tell whether particular 
impacts can be attributed to the project 
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Public policy education projects need to include self- 
refleciion and careful documentation of decisions as 
evaluation methods. Projects in die cluster were probably 
engaging in main* such discussions but not necessarily 
viewing them as evaluation or recording diem in a form 
that was helpful to them in their evaluation efforts. 

LESSON 16 

Several viable staffing models were used for evaluation, 
but project staff were often the sole cvaluators of their 
projects. That practice should be reduced in future 
projects. 

The projects adopted four different staffing models to 
evaluate their endeavors; 

1. Project staff as evaluators. Project staff often 
developed and administered instruments for participant 
feedback and conducted ongoing informal process 
evaluations as their projects developed. 

2. Internal coalition committee as evaluators. A 

committee developed the evaluation instruments and 
provided ongoing evaluative feedback to project staff. 

3. External evaluators. External evaluators were con- 
tracted to develop all evaluation strategies, conduct 
evaluations, analyze the findings, and report findings to 
the coalition. 

4. Combination of external and internal evaluators. 

In one project, external evaluators were contracted to 
develop and implement an evaluation strategy lor project 
events during the project's first year, and that strategy 
was then replicated by project staff in later events. Other 
projects hired external evaluators to conduct outcome 
evaluations but conducted ongoing project improvement 
evaluation themselves. 

Although we believe that many approaches can be 
viable, we concluded that project staff should not be the 
sole evaluators of their projects, riven if they are able to be 
objective, project staff do not have enough lime to evaluate 
their projects adequately. I'uture projects should include 
enough money in their budgets to elicit evaluation assis- 
tance separate from project staff, even i I it is internal to the 
organization or coalition. 



LESSON 17 

Impacts on participants in project activities were reported 
far more frequently than impacts on public issues or on 
the policy-making process, even though the latter were 
clearly of interest to the educators. In the future, more 
emphasis should be given to the assessment of issue or 
process impacts. 

The projects generated an impressive and diverse array 
of reported outcomes. Throughout their evaluation reports, 
other project documents, and our own cluster evaluation 
interview logs, many statements described project out- 
comes. Some came from participants responses to evalua- 
tion instruments: some were self-reports by individuals we 
interviewed: others were impressions or observations by 
project staff. 

Analysis of these statements yielded forty-live categories 
of outcomes, the majority of which were participant 
outcomes. Most were impacts on project audiences, but 
some were impacts on coalition members. Most of the 
audience outcomes were references to individuals, but 
some were impacts on groups or organizations. Examples 
of participant outcomes included 

• increased confidence (less intimidated by experts and 
prominent people) 

• intent to network with others in attendance 

• increased awareness of the persistent problems our 
environment faces 

• recently appointed head of a state agency commented 
that she learned a lot preparing for her role as a discus- 
sion leader on an issue that "was not my background at 
the time or an area of interest"; later, she was able "to 
handle these issues better in setting policy recommenda- 
tions for the governor" 

• increased knowledge about groundwater 

• increased overall economic intelligence of rural and 
urban writers and editors who cover issues related to 
agricultural and rural policy 

• understanding of the shift in emphasis from production 
and marketing to environmental concerns in the farm 
bill 



Who knows how long it would take to pot this kind of expo- 
mho on my own! 

-study group participant in one of the project 

It like listening to d ludio talk show, listening to owiyono'i, 
opinion, but no cRtion wds tdken. 

-participant in a project's roundtable discussionss 
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• sense of ' hopefulness" among ihe group at the end that 
they could, in fact, influence policy 

• "made us examine some of our silly competitive tenden- 
cies" 

• "I can better appreciate the farm point of view and 
recognize that farmers arc concerned about man)' of the 
same issues that I am" 

• "I think I have become more tolerant, since 1 realize this 
is a complex problem and not one that will change 
overnight" 

• a county supervisor became more interested in develop- 
ing recycling programs: he had opposed the environ- 
mental movement but now realizes that problems have 
to be dealt with 

• "people outside the agriculture community arc more 
open-minded about environmental problems than I 
thought" 

• became very worried that it is harder to bring views 
together 

• 86 percent shared the materials with others 

• in waste management, people who attended the down- 
links have taken the lead in getting their local govern- 
ments to comply with the state law 

• got on a public issues committee as a result of one of the 
programs 

• one participant was instrumental in reversing a commu- 
nity decision preventing out-of-county waste in the 
county landfill after learning about the economics of 
regional land filling 

In addition to participant outcomes, there is an assump- 
tion in public policy education that changes at the indi- 
vidual level will result somehow in "better" decisions. That 
assumption suggests the importance of also examining 
actual impacts on policy-making, and a substantial number 
of issue or process outcomes were in fact reported in 
project documents and interviews. These differed from 
participant outcomes bee ause they concerned changes in 
the issue or process rather than changes to particular 
individuals, groups, or organizations. 2 Examples included 



• increased visibility of the issues by producing high- 
quality publications disseminated to thousands of people 

• involving outlying areas and people not usually involved 
in activities related to these issues 

• issues reported from a broader perspective by news 
media 

• rural health, rural education, agriculture, and industrial 
recruiting interests are not yet part of the same lobby, 
"but were certainly moving in that direction" 

• the state office for rural health was established much 
more quickly than if the conference had not been held 

• two school districts that had originally been antagonistic 
viewed the program together and are now discussing 
cooperation 

• congressional legislation has been drafted for $40 million 
worth of R&D on the trade corridor and border crossing 
impediments; this development can be traced directly to 
Anderson s research, the need for which was identified at 
the 1991 conference 

• language (on groundwater protection) drafted for the 
county plan is expected to be adopted 

• since the program, three more communities in the 
county have started curbside recycling 

The predominance of participant outcomes reflected a 
traditional view of education as something that happens to 
individuals (or to separate groups or organizations) and not 
as an interactive community activity. It also reflected the 
absence oi evaluation strategics aimed at assessing impacts 
on anything other than individuals. Despite this absence, 
several projects reported such impacts, usually in the form 
of anecdotal ev idence. We viewed this evidence as an 
acceptable data source that could have been used more 
strategically. For example, when a story of such an impact 
reached project staff, it would have been helpful to trace it 
to its source and to ask questions such as, "Wha* was it 
about the project event that contributed to this impact on 
the issue?" or "What would have happened with respect to 
this issue if the project event had not occurred?" Systemati- 
cally documenting these responses not only could establish 



I In- distinction between p«n tu ip.itii .uul issue oi process mi iconics was 
siuih link s .unln^uoiis l or c\.mipli\ one pio|c*c t lepoiled i omiuumU benehis' 
sin h as nu leased coininilinenl lo mu liu^. Inn n was uiu leai \\ Ih iIh i these weie 
best iliou^hl ol ,is se pat .lie mcli\ ichi.il at nous or .is ,i ^eiuiiiu- tolled iw m 
community phenomenon, relic* led. toi example, in nu teased sitppoii lor 
let \t Inn; pto^i.niis Aiinilu-i (-sample w onld In- ( lunges in .intitules u>w,ncl issm s 
oi polic\ posiiions. itoi In I .ink .ind lile c nueiis, bin b\ .1 kc\ polu \ makei Is thai 
"itttreK '.in individual panic ip.int outcome oi an unpad on lite pnlnv making 
piocess? I oi tuuaieh . sue h ambiguities weie the c m eptiou ralltet ih.iu lite mlr so 
lite distittc lion was not Iwrcl lo make in uiosi cases 



M<iybe wore d little more lesults oriented thin you drive foi nt 
cin cKcKleinir institution. 

—project participant a small-town business person 

Uy the end of the uc?y, they I uul -.oinc common philosophies 
some common beliefs. . .suddenly, rill these walls were 
( oming down. 

-project staff, describing one of the project's study groups 
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Vignette: Issue oify^ 

from Two Conference in Montana y 

Two conferences held by the Upper Midwest project 
in Montana-one on United States-Canada trade and 
one on rural health care-were said to have had im- 
portant impacts on polio/ decisions in the state. The 
cluster evaluation team visited Montana to learn about 
issue or process impacts and how they occur. We 
talked with several individuals who either helped plan 
the conferences, participated in them, or were in- 
volved in follow-up activities. We asked the following 
questions: 

What happened as a result of these conferences? 

In the case of United States-Canada trade, people 
reportedly left the conference realizing they needed to 
"get serious" about free trade and committed to "take 
it [to] the next step." The Montana representative on 
the project's coalition said, "People were cornering 
me, saying, 'We've got to do something.'" A Canadian 
official offered to host another conference. A steering 
committee was established.; further research was 
commissioned; a second conference was held; and a 
third one was being planned at the time of our visit. 
People from all over the West were calling to see if 
they could attend. And a member of the Montana 
Congressional delegation expressed interest in the 
research and subsequently introduced a bill to fund 
trade corridor and border crossing improvements. 
"You want an outcome?" the coalition representative 
said. "If that legislation passes, I don't know of another 
thing that would have more impact on the well-being 
of Montana!" 

Regarding rural health care, we were told that a 
State Office for Rural Health and a Montana uural 
Health Association were being established as direct 
outcomes of the conference. A bill to provide state 
funding for the Office for Rural Health was introduced. 
Although it did not pass, it was expected to be reintro- 
duced in the next legislative session. 



What was it about the conferences that led to these 
results? 

Regarding United States-Canada trade, a speaker's 
presentation on a successful trade corridor in the Red 
River Valley apparently provided a concept that people 
could rally around. Timing was also critical. Inter- 
viewees said several related things were happening 
around the time of. the conference. For example, a 
recently ratified free trade agreement was of great 
interest and the governor had been focusing attention 
on Montana's trade relations with Canada. The 
conference brought together the "right mix of people," 
and the speaker on the Red River corridor "planted 
the seed." The number and mix of participants was 
said to be critical, helping convince participants that, "if 
all these people are interested," the concept must be 
important and has a good chance of success. 

As for rural health care, when three discussion 
groups at the conference were asked to develop 
policy recommendations, all three identified an office 
for rural health as their first recommendation. 

What would have happened if the conferences had 
not been held? 

In the case of United States-Canada trade, the 
people we talked with said promotion of one-on-one 
exchanges between U.S. and Canadian business 
people, which had already been underway, would 
have continued, but the trade corridor concept as 
such would not have come into focus. The confer- 
ence gave rise to a more formal structure to promote 
trade corridor improvements and an identifiable group 
of people working on it. 

Regarding rural health care, interviewees said the 
Office for Rural Health and the Rural Health Associa- 
tion would have been established anyway, but it 
would not have happened so quickly. The conver- 
gence of all three discussion groups on the same 
recommendation "gave us advocacy we could use" 
and thereby speeded up the decision-making process. 
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connections between the project as a whole and impacts on 
the issues, but could help discover specific aspects of a 
project that would have the most impact on the issues. 

A second suggestion for evaluating project impacts on 
the issues is to involve people who are already in a position 
to observe such impacts. For example, several projects 
established advisory groups composed of people involved 
in the issue under consideration. It would have been a 
natural extension of those groups' involvement vvith the 
project to conduct focus group interviews afterward to get 
their impressions of the projects impact on the issues. 
News reporters covering the relevant topics nr also 
provide helpful observations about possible i . • on the 
issues. 

Paying attention to impacts beyond those on individual 
participants is important. Although documenting them can 
be a slipper)' endeavor, we advocate using and extending 
available resources to maximize the quality of data on these 
topics. Careful thought could be given to the questions to 
be asked. For example, knowing that a project influenced a 
particular decision may be less useful than knowing 
whether the resulting decision was based on more informa- 
tion, was participated in by greater numbers of people, or 
was acceptable to a wider range of interests than would 
likely have been the case in the absence of the project. More 
systematic or carefully collected information on what is 
happening to the issues or the process, as well as to project 
participants, can be useful for program management 
decisions. It should also be helpful in addressing the often 
troublesome accountability questions about ih° impact or 
value of public policy education. 

LESSON 18 

Issue or process outcomes were more likely to occur 
when the scope of a project's audience corresponded with 
the policy arena in which the issues addressed by the 
project were resolved. More attention should be given to 
this relationship in future projects. 

The projects varied strikingly in their tendency to report 
issue or process outcomes. Five projects reported numer- 
ous such outcomes, whereas six did not. Some projects 



reporting issue or process outcomes focused on issues 
decided in local policy-making arenas such as groundwater 
decisions in Winnishiek County, Iowa. This caused the 
leaders of one project to conclude that issue or process 
outcomes were more likely in projects with a local focus, 
where the scale of the project belter matched the resources 
available within the time frame of the project. According to 
this reasoning, comparable impacts in the state or national 
arena would have required a longer time frame — and, 
hence, sustainability beyond the period of Kellogg funding. 

Other projects, however, demonstrated that issue or 
process outcomes were also possible in the state policy- 
making arena. One project reported impacts on state rural 
development policy, another on state decisions regarding 
health care and United States-Canada trade. This suggested 
that the difference was not simply whether the focus was 
local, but whether the project's audience included a cross- 
section of participants involved in the appropriate policy 
arena. The project that reported issue or process outcomes 
in rural development included a cross-section of state rural 
development interests in its audience, whereas the one 
reporting impacts on health care and United States-Canada 
trade decisions involved similarly appropriate cross- 
sections of state health care and trade interests. 

Contrasting situations — in which issue or process 
outcomes were less likely — were illustrated by two projects 
whose participants learned about issues in the national 
policy arena, but then returned home to work either on 
state or local issues related to the national ones or on the 
national issues themselves. In either case, the impact of the 
projects at the issue or process level was likely to be 
diffused. For example, in one project whose focus was on 
the national farm bill, participants might return home and 
attempt to influence policy-making in Washington. But. 
regardless of how much they were affected by the project, 
they would still represent only a tiny fraction of the influ- 
ences on the farm bill. Alternatively, thev might return 
home wishing to apply their new know icdge to state or 
local issues but might have little impact because no one else 
in the same slate or locality had participated in the project. 
In either case, individual learning as a result of the project 
might be substantial, but issue or process impacts would 
likely be negligible. 



At the minimum, we increase the information out there. But 
we don't really know if an increase in good information 
actually leads to greater involvement and participation — We 
really don't know the impact of the project in the trenches. 

-project leader 



It was a surprise for the environmentalists that there were 
farmers who shared their goals. 

—extension specialist who participated in a 
project's roundtable discussions 
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The implications of iliis finding w ere not completely 
clear. On the one hand, issue or process outcomes were 
more likely if a projects audiences corresponded with the 
policy arena in which the issues addressed were resolved. 
State issues or policy-making processes could he affected by 
projects with statewide audiences but not by those whose 
audiences were coiifiiud to a single county or region of the 
stale. Likewise, national issues or processes could be 
affected, but only by projects with a national audience. We 
did not assume, how ever, that projects without such a 
correspondence of audience and policy arena should never 
he undertaken. It is possible that projects seeking to reach 
ordinary citizen* or to mobilize new participants in the 
policy-making process ean best enhance their effectiveness 
by working at the local level. (That conclusion was reached 
by more than one project in the present cluster.) In that 
sense, it would often be appropriate for state or local 
projects to educate about national issues — or for local 
projects to educate about stale issues. But such projects 
should have realistic expectations regarding outcomes at 
the issue or process level, at least in the short run. Starling 
at the local level to influence national policy could be a 
worthy and perhaps necessary strategy but would likelv 
require sustainability beyond the normal three- or four-vear 
time frame. 

LESSON 19 

Different outcomes appeared to be associated w ith 
different stages of development in the issues addressed In- 
die projects. Future projects should devote more atten- 
tion to this relationship. 

Our data revealed at least a modest tendency for the 
outcomes reported by particular project* to cluster in 
different stages, but the patterns were not clear enough to 
make much sense of them. Not all participant outcomes 
were di recti) related to public policy. Some, such as 
expanding ones personal knowledge, making contacts. 01 
learning things that were used in ones work, might have 
little or no consequence for public policy. In addition, a 
signilicant number ol the reported participant outcomes 
were oriented toward educating others rather than direct 1\ 



affecting public policy-making. But most of the remaining 
participant outcome categories could be arrayed in a 
sequence that roughly approximated a series of steps or 
stages in the policy-making process (Tabic 1 ). Although the 
placement of individual items in such a list was sufficiently 
arbitrary that differed people would probably come up 
with different lists, tl re would likely be considerable 
agreement, at least on the general pattern. 

Projects varied in their tendency to report outcomes 
related to different stages. Only three projects reported 
participants gaining awareness of problems or issues. 
Nearly all had reports of participants gaining know ledge. 
Several reported increases in participants' motivation or 
commitment and instances of people learning about other 
perspectives. Finally, most projects had reports of partici- 
pants attempting to influence decisions. One might like to 
think that each project emphasized outcomes that were 
appropriate given the stage of development of the specific 
issue or issues being addressed, but we lacked sufficient 
data about the issues themselves to explore such a possibil- 
ity very thoroughly/ 

The issue or process outcome categories could also be 
arrayed in a sequence corresponding roughly to stages in 
the policy-making process (Table 2). Different projects 
emphasized outcomes in different stages, but. as in the case 
ol participant outcomes, the patterns were not exception- 
ally clear. Of the five projects with significant numbers of 
issue or process outcomes, three reported outcomes 
throughout the policy-making process, whereas one 
emphasized the earlier stages of increased awareness, 
creating new organizations, and increasing participation, 
and another emphasized the later stages of increasing 
interaction. de\ eloping proposals, and seeing legislation 
enacted. 

Although there is not much evidence on which to base 
conclusions, these last two cases were consistent with the 
objectives ol the respective projects — one to empower 
previously uninvolved people (hence emphasizing early 
stages of policy-making), the other to broaden the influ- 
ences on rural development policy-making < mphasizing 
the later stages). 



The fcirmois wore siifj <:d tluil \\v iveicn't <i!l fLiminn 
rjdiCcilr-. 

— environmentalist who participated 
in the same discussions 
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TABLE 1 

Participant Outcome Categories Arrayed in 
Rough Order of Stages in the Policy-Making 
Process 

Personal growth, personal knowledge, gained skills or 
confidence 

Used in ones own work something thai was learned 

Made contacts, networked, began working together with 
others 

Gained awareness of problems or issues 
Gained knowledge about an issue or situation 
Learned about the political process 
Saw that one can make a difference 
Motivated to take action 

Saw need for more information or education — for sell 

Saw need for more information or education — for others 

Learned about or saw the value of working together 

Discussed with others something that was learned 

Became more committed 

Learned about other perspectives 

Learned about other perspectives and saw more conflict 
than expected 

Learned about other perspectives and saw more room for 
agreement 

Engaged in communication between citizens and 
policymakers 

Gained broader view of an issue or saw connections 
between issues 

Formulated a personal position on an issue 

Changed one s ideas about an issue 

Took some kind of action but of a nonpolilical nature 

Educated or shared information with others 

Saw the potential for long-term resulls 

Attempted to influence decision making on an issue 

Joined a board or committee 

Successful lv influenced a decision 



TABLE 2 

Issue or Process Outcome Categories 
Arrayed in Rough Order of Stages 
in the Policy-Making Process 

Increased awareness of a problem or issue 

Stimulated action on an issue or the creation of new 
organizations 

Increased participation in an issue 

Information u>ed by participants in the political process 

Influenced media coverage of an issue 

Increased interaction among diverse interests 

Increased consensus, common understanding, or shared 
sense of urgency 

Structure developed for taking action on an issue 

Meetings held to investigate or resolve an issue 

Moved an issue onto the political agenda, attracted more 
attention by policymakers 

Legislative proposal developed 

Legislation enacted 

Program implemented 
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It seemed to us that project objectives and implementa- 
tion strategics should "fit" with the stage of development of 
the issues being addressed. Careful though' along these 
! ncs should be a part of program planning. Evaluation 
should J^o focus on the match between objectives and 
strategics v., - the one hand and the stage of development of 
an issue on the other. Defining project objectives as helping 
to move participants or an issue along the sequence of 
stages in the policy-making process might be a useful 
alternative to the typically frustrating challenge of thinking 
or talking intelligently about educational program impacts 
on the ultimate resolution of an issue. 

LESSON 20 

Attention to capacity building within public policy 
educators' own organizations is another important 
consideration, in addition to participant and issue or 
process outcomes. 

A third and final major group of outcome categories 
included capacity-building outcomes for public policy 
educators themselves. Reported impacts on coalition 
members were included here if they involved changes in 
the capacity to earn - out public policy education effectively. 
These outcomes fell into five categories: 

1 . building relationships with other key actors such 
as the governor" s office, relevant state agencies and 
interest groups, or other educators (e.g.. relationships 
between C ooperative Extension and the League of 
Women V oters); 

2. developing knowledge, skill, experience, or 
confidence— including demonstrating to colleagues in 
extension that it is possible to conduct unbiased educa- 
tional programs on controversial issues, drawing subject- 
matter specialists into the public policy arena, or 
increasing League of Women Voters representatives 
understanding of agricultural audiences: 

3 enhancing visibility or reputation or building 
broader audiences; 

4 influencing or assisting other educational efforts 
in the rJucators' own organizations, such as 



promoting attention to certain topics, serving as a model 
for other educational programs, or providing useful 
resource materials; 

5. influencing related developments in the educa- 
tors' organizations, such as selecting issues for 
extension to focus on, preparing field staff for the 
teamwork needed in "issues programming." or dev elop- 
ing proposals for new projects. 

Considerable attention was given to such outcomes in 
project documents and in our interviews. Such attention 
was not surprising given the limited development of public- 
policy education as a field of practice and the consequent 
need lor educators to work on their own long-term capacity 
to implement public policy education at the same time as 
they attend to the immediate requirements of particular 
educational programs. 

Although accomplishment of capacity-building out- 
comes and of participant and issue or process outcomes 
ought to be com pic me mar)', there was a danger of paying 
too much attention to one dimension at the expense of the 
other. On the one hand, educators could pay so much 
attention to whether their programs were I i keel or respected 
that the program's educational quality might suffer. On the 
other hand was the danger of implementing excellent 
programs without, in the end. enhancing the capacity to do 
similar programs on other issues or to build on a one-shot 
success. The latter was particularly likely when special 
temporary staff were hired to run a program. A balance 
must be sought between these extremes. 

LESSON 21 

Realistic and significant targets for sustainability from 
public policy education endeavors are changes in the way 
participating organizations understand, value, or conduct 
their work (i.e., capacity-building outcomes). Creating 
new organizational legacies is a legitimate secondary 
purpose of public policy education projects and should be 
a primary target for sustainability. This, in turn, requires 
that the institutional participants in the projects be 
learning organizations. 



I'm proudest of the fact ttvit we generated a new audience fot 
extension. 

—project leader 
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Like most externally funded applied social science 
programs, one requirement of project funding, reporting, 
and accountability in the present cluster of projects was 
project su$tainahilit\\ that is, a project's ability to continue 
in some form alter the cessation of external funding. 

For public policy education projects, however, the 
concept of suslai nubility presented two problems. First, it 
could be argued that the most successful public policy 
education projects, once completed, were no longer 
needed. Successful projects accomplished their educational 
mission, thereby either invoking a new and different ..?cd 
for education, catalyzing action, or shifting the grounds and 
values on which the issue was being debated. Thus, in 
many specific cases, sustainability of the project itself was 
neither warranted nor desired. Second, the boundaries 
between aspects of the projects that were sustained and 
project outcomes — particularly outcomes other than 
impacts on participants — were ambiguous. For example, 
was increased interaction among diverse interests on a 
given issue a process outcome, an important aspect of the 
project that was sustained, or both? And was an 
organization's increased desire and ability to work 
collaboratively with potential antagonists a capacity- 
building outcome, an important project legacy (under the 
heading of sustainability). or both? 

These questions arose during our analysis of project 
sustainability. With further reflection came the insight that 
changes in organizational legacies might be more important 
th an perpetuation of a particular project. 

What kinds of things were reported as sustained? 
• project materials. At least four projects pointed to the 
materials they developed as a significant legacy. Coali- 
tion members in one project were particularly proud that 
their materials have a iong shelf life." This project, 
however, had considerable difficulty disseminating its 
materials. A long shelf-life has little value if the materials 
remain on the shelf. In contrast, in another project, one 
set of materials had to be updated between final draft 
and publication dales. And in a third project, the 
materials appeared to be valued as much for their role in 
legitimizing the piojeet as in disseminating educational 
information about the issues. 
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• new organizational legacies (capacity-building 
outcomes). Such legacies could be subdivided into 
three groups: 

1. Changes in the structures organizations use to 
envision, organize, or administer their work 

~ One projects coalition was sustained .s a standing 
committee of a professional organization, thereby 
ensuring continued state-level attention to educa- 
tional needs on water policy issues 

- Working relationships between two participating 
organizations continued beyond a project 

- New working relationships with the media were 
established during a project and continued afterward 

- Members of a projects coalition developed a commit- 
ment to work as a coalition again in the future 

2. Changes in how the work of organizations gets clone 

- One projects educational process (promotion of 
dialogue via roundtablcs) was adopted by other 
sectors of one of the participating organizations in 
their educational outreach work 

- Project materials were incorporated in existing 
curricula and activities of the main participating 
organization in the project 

- Project materials were subsequently used by teachers 
who participated in the project s conferences in their 
own classrooms 

3. Changes in what work gels done 

- A participating organization continued its newly 
directed attention to agricultural issues beyond the 
lifetime of a project 

- A new professional development opportunity created 
as part of a project for the members of one participat- 
ing organization was continued after the project was 
over 

- Local capacity was developed to continue activities of 
a project (e.g.. state and local workshops), and partial 
funding was secured for project continuation 



i guess it's kept me thinking about public policy education and 
how important it is. I'm even more adamant in having people 
understand that things a;e multifaceted 

—extension consumer science specialist who 
collaborated with one of the projects 
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All of these examples represent the potential for one or 
more participating organizations to do things differently 
after the grant period ended — to work collaboratively 
rather than alone, to include rather than exclude particular 
issues on their agenda, to consider previously ignored 
groups as part of their target audience, to engage in open 
dialogue before making decisions and commitments. In a 
public policy education context, such potential changes in 
how organizations do their work were significant. More- 
over, they represented realistic legacies from a single 
project effort. 

Strong, healthy organizations brought existing legacies 
to an adventure such as public policy education. Used 
thoughtfully, these legacies could help ground new. 
creative, innovative, risk) ventures in practices with an 
established record of success. Beyond building on strengths, 
forward-looking organizations also engaged in innovative 
learning — learning capable of bringing change, renewal, 
refructuring, and problem reformulation (Bcnmsand 
Nanus 1985). Innovative learning represented the creation 
of new organizational legacies. It might have been ex- 
tremely difficult for a public policy education project staff 
to determine in advance what new organizational legacies 
would be created in a project. Yet many projects were able 
to articulate desired changes for coalition members during 
the beginning phases of their work. Such articulation 
signaled interest in innovative learning — in new legacies — 
and such interest could be sought in advance. 



I he pl<Jte pcissod oiound and wo didn't put anything in it. 
I felt bcul. 

-coalition member, commenting on lack of 
funds to continue the project 

We'll he hi-re J lonp, time dller Kellogg's just a com lldke dgdin. 

-coalition member, commenting on the 
expected longevity of their coalition 
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THE IMPACT OF DIALOGUE 



A CASE STUDY 



The conversation started slow ly and awkwardly. Jcnnv 
Carpenter, the discussion leader, read the first question: 
"Should sound environ menial practices be required for 
farm program participation?" After some uncomfortable 
silence. Ben Gcarharl. a grain and livestock farmer, said. 
"We're already using them. It s driven some of the poorer 
farmers out of business/' No one else spoke. A resource 
economist from the university, in an effort to be helpful, 
suggested expanding the question to refer to additional 
environmental constraints. Charlie Harrison, an agricul- 
tural agent w ith Cooperame Extension, mentioned LISA. 
Jenny asked lor a definition, but no one replied. A represen- 
tative ol a food processors' trade association said farmers 
used to do more environmental damage than they do now. 
and Harrison told a story about someone using diesel fuel 
to kill weeds in a church parking lot. The conversation 
wasn't going anywhere. 

Ten individuals had gathered in a hotel meeting room 
lor a roundtable discussion on the environment. It w as a 
diverse group — Jenny; the economist from the university; 
two leaders from the League of Women Voters; Gcarharl 
and another farmer; the extension agent; an officer of the 
Sierra Club; the trade association representative; and a 
secretary to take notes. 

Jack Kinney, the second farmer in the group, said he 
thought sound environmental practices should be required, 
but the requirements need to be easy to understand. Jenny 
tried again to get someone to define LISA, but the 
procnvironmcnl statement from Kinney, a farmer, had 
apparently initiated a shift in the conversation. Linda 
Goldstein, one of the members of the League of Women 
Voters, was the first environmentalist to speak up. She said 
she thought the issue w as broader and invok ed implica- 
tions for the larger environment. Gcarharl rephrased her 
comment to make sure he understood it. and then Kinney 
said. "We need education Farmers can sec soil erosion, but 
we can't see chemical runofl. If we did, w e'd be belter 
stewards. I'm wasting money if the chemicals wash a\va\ ." 
Goldstein, troubled by what seemed to be a narrow! v 
economic outlook, said. There are groundwater risks, too. 
not just financial ones!' And then Gcarharl jumped in 
again. "Farmers arc consumers, too." he said. "Mv well is 
only 12 (eel deep. My family drinks that water! Pon t think 



I'm not concerned if my farming practices contaminate the 
groundwater!'' 

Kathy Olmstcad. another environmentalist, then spoke 
up. "Farmers know more about pesticides and soil erosion 
than I do. But w hat I'm concerned about is how to make 
sure that compliance is going to happen. I agree with the 
importance of protecting properly rights, but the rule has to 
be thai the farmer can't do harnrihat extends off the farm/' 
Kinney, implying he'd heard that argument before, said. 
"Okay, but I si ill say the rules have lo be specific. We need 
clear direction." Olmsiead. sensing a confrontation, backed 
off a bit — saying. "I was just throwing something oui" — 
and the conversation turned lo other topics. 

Before lime ran out. ihc group went on to talk about 
participation in commodity programs, alternate cropping 
systems, ihc pros and cons of buying locally, and the 
conservation reserve program. Throughout the conversa- 
tion, they continued to be careful with one another — 
verbalizing agreement with individuals on the other side of 
the issue, saying things in joking ways, and backing off if 
anyone seemed to be getting angry. But they were not just 
placating one another. Individuals on both sides were 
adamant about certain points such as the unacceptability of 
unclear regulations to the farmers and the environmental- 
ists' determination to keep pressing for control of agricul- 
tural pollution. Scattered conversations continued in the 
meeting room and hallway as the meeting broke up. 



The roundtable discussion was part of a series sponsored by 
a project in Illinois called Agriculture and Food Policy in an 
Interdependent World. The project was carried out by 
Cooperative Lxtension and the League of Women Voters. 
Roundtable discussions were held in fall 1089 at four 
locations around the state, covering issues related lo the 
l c ) c )0 farm bill — international trade, commodity programs, 
and lood programs as well as the environment. The project 
also sponsored a statewide follow-up conference a year 
later. The goal was to losler dialogue about food and 
agriculture issues between rural and urban citizens. 

Such dialogue docs not happen by accident. In this case, 
extension and the league were ideal partners to make 
something different happen. "We could not have done it 
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wiihoiu ihc league." said Sandra Erikson. ihc project leader 
from extension, a faculty member in agricultural economies 
ai ihc slate university. "1 had good knowledge of the 
agriculture community and contacts in downstate Illinois. 
Megs contacts with nonfarni organizations and in the 
Chicago area were invaluable, and the league's name helped 
promote the project." These complementarities were useful 
in forming a steering committee for the project, recruiting 
discussion leaders lor the roundtables. and ensuring the 
desired mix of roundtable and conference participants. 
Getting adequate nonfarm participation was one ol the 
biggest challenges. Meg said. "I made 20 phone calls for 
every nonfarni person who actually attended."' 

Extension and the league — Sandra and Meg — also 
complemented one another in their mix of technical 
knowledge and process skills. Organizers decided early on 
that the project should concentrate on facilitating commu- 
nication between farm and nonfarm interests, but they 
recognized that a certain amount of information also 
needed to be provided. Meg admitted to knowing little or 
nothing about agriculture whin the project began, but she 
had a background in speech and communication and had 
developed excellent process skills in her work with the 
league. ' Sandra developed the background materials." Meg 
said, "while I worked on design of the meetings and on 
recruiting discussion leaders." Keynote and luncheon 
speakers were also selected lor the roundtables. and a 
resource person was assigned to each discussion group. The 
follow-up conference also emphasized the provision ol 
information through speakers and panels. On the process 
side. Meg also prepared a handbook and conducted an 
orientation session for the discussion leaders, and a break- 
out session for small-group discussion was included in the 
conference. 



What impact did the project have? Ml had a great impact on 
me/' said Lihbv Becker, the recently appointed head ol a 
slate agency. She had been a discussion leader at one ol the 
roundtables. "1 was a discussion leader lor a group on 
environmental issues. That had not been my choice. I was 
more interested in international trade. But I learned a lot in 
preparing lor the discussion and also from the dillereni 
participants in the roundtable. especially the environmen- 
talists, i then became a member of the governor's transition 
team, serv ing on a committee that included the environ- 
ment. 1 was able to handle issues better in selling policy 
recommendations lor the gov ernor. Now thai I'm in slate 
government, we have environmental programs in our 
agency, and we deal with other agencies and the legislature 



on environmental issues. 1 would not have been as well 
prepared if 1 had not participated in the roundtable. I am 
more open lo other sides of environmental issues than 
oilier directors of my agency have been, and the roundtable 
definitely had an influence on thai." 

Project evaluations indicated that participants in the 
roundtables and conference gained useful information from 
the background materials and speakers. They also fell that 
participation was worthwhile for hearing others" opinions 
and thai, their own perspectives on the issues had been 
affected. Both environmentalists and farmers indicated they 
had learned something about the oilier side. Some were 
alarmed to find the disagreements bigger than they ex- 
pected, whereas others became more optimistic about 
prospects for mutually agreeable solutions. A prominent 
speaker at one of the roundtables said he was going to 
inform the U.S. secretary of agriculture about the degree of 
interest in environmental issues demonstrated at the 
roundtables. and a congressional staff person had previ- 
ously told Sandra, if you can get farm and environmental 
interests to agree on anything, we d be very interested." 

But Kathy Ol instead said the project had no impact on 
her at all. "1 am from an Illinois farm family. I have some 
experience with these issues. I know what farmers are 
doing out there. And Ini a fierce advocate for environmen- 
tal law and regulation. What was I going to learn? Why was 
this dialogue going to change anyone's mind? liven if I were 
going to change my perspective, one person from my 
organization would not change the organization's position." 
Kathy had been a member of the project s advisor)' commit- 
tee. "I had advocated lor a more aggressive, clearer goal 
than education. The goal should have been to solve a 
problem — to come up w ith a new, joint perspective. They 
were li v ing to foster dialogue, but not lo foster a solution. 
People at the events were not going to change. To just have 
dialogue isn't worth it. just for the sake ol talking. The 
roundtables w ere not reaching out to a general audience. 
The players involved had set positions already. The project 
should be bringing in politicians, who may not have the 
breadth and depth of know ledge on the issues." 

Others shared some of Kathy s dissatisfaction. Some said 
there was too much sharing of ignorance in the 
roundtables; others, that many participants were ahead) 
knowledgeable and those who really needed lo learn were 
not present; still others felt that not enough would really 
happen as a result of the events. For most, the positive 
value ol the events outweighed the failures, but their critical 
comments, like Kalhv's. are food for thought for future 
projects. 
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CONCLUSIONS 



The projects that we studied w ere part of an ongoing 
process of helping educators in extension and other 
organizations to facilitate the understanding and resolving 
of contentious public issues. Our purpose in this conclud- 
ing chapter is to fit the lessons learned from these projects 
into this ongoing process. What advances have been made 
since these projects were completed? What further lessons 
have been learned? What gaps and needs remain? 

• Coalition building and maintenance (Chapter 2) 
Building and maintaining coalitions, especially regard- 
ing education about public issues, is currently receiving 
a lot of attention (Cooley. Duncan, and Burridgc 1994: 
Public Issues Education Materials Task Force IW). The 
distinction between educational and political coalitions 
(initially pointed out by staff in the Groundwater Policv 
Education Project) has been sharpened. Political coali- 
tions whose memberships represent multiple perspec- 
tives ( 'consensus-seeking" coalitions) arc considered 
desirable for public issues education, w hereas more 
narrowh' based political coalitions ("advocacy" coali- 
tions) arc considered hazardous to effective, balanced 
education (Dale 1993; Hahn 1994; Public Issues Educa- 
tion Materials Task Force 1994). As Dale (1993) points 
out. educational coalitions with membership inclusive of 
all points of view on an issue sometimes have mutual 
education as a goal (similar to or the same as consensus- 
seeking coalitions). 

We argue, however, that educational coalitions do not 
need to be inclusive to be useful. Moreover, they do not 
even need to exist in the formal sense, as long as the 
resources they provide can he obtained in other ways (as 
discussed in Chapter 2). What is critical is the ability to 
implement educational programs that are fair and 
respectful of divergent interests and viewpoints, tap the 
needed resources, and communicate and work with 
di\ "crse individuals and organizations. 

• Implementation of educational programs (Chapter 
3) Several themes related to implementation continue to 
be important: 

I . An immense amount of attention has been paid to 
dialogue in recent years. Major efforts have been made 
to provide extension educators with knowledge and 
skills related to dispute resolution, including a 
national videoconference. a workshop at the 199 3 
National Public PoIic\ Education Conference, and 
inclusion in new printed materials lor in-service 
education (Sachs et al. 1993: Danielson and Carber 



1994; Eaas 1994; Jones 1994; Public Issues Education 
Materials Task Force 1994). Other dialogue formats 
(Dale 1993) receiving attention by extension educa- 
tors are the Kettering Foundation's National Issues 
Forums (Garkovich 1994); study circles as promoted 
by the Study Circles Resource Center in Pom fret. 
V ermont; and the 'Citizen Politics" approach advo- 
cated by Many Boytc and Project Public Life at the 
University of Minnesota. Increased attention is also 
given to the process of discussion and issue resolution 
as an alternative or supplement to content, as well as 
continued reminders that educators should not 
overlook content in the rush to learn process skills 
and techniques (Flinchbaugh 1994). The emphasis on 
dialogue and process is seen as a way of extending 
educators' roles beyond the traditional alternativcs- 
and-conscqucnccs model to include assistance in the 
decision-making and implementation stages 
(Danielson and Garber 1994; Faas 1994). 

2. Empowerment is frequently discussed, although not 
always with the same meaning we gave it in Chapter 
3. The notion of extending the educators role into 
decision making and implementation is sometimes 
referred to as empowerment. In this view, the three 
"modes" of public policy education that we identified 
in Chapter 3 arc treated as a sequence of developmen- 
tal steps — information provision, followed by dia- 
logue (the chance to exchange views), and then 
empowei , m (taking action). Although such an 
interpretation makes sense, we believe it diverts 
attention from the more important point that we 
wanted to make. Our definition of empowerment — 
the involvement of people affected by public issues but 
neither invoked nor adequately represented in the 
policy-making process— appears to be a topic on which 
there is too little experience among public policv 
educators and too few lessons. 

3. Too little attention is given to the role of c ontent. In 
Chapter 3. we argued that content as well as process 
was important in education about public issues. I lere 
we add that content should be incorporated in 
dialogue and empowerment modes and not confined 
to information provision. Lessons are needed lor 
content specialists — educators who are experts in the 
subject matter ol particular issues: many of the 
project leaders were content specialists rather than 
"process tvpes" (such as leadership development or 
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community development specialists). Important 
lessons were undoubtedly learned by those individu- 
als on how to provide content in the context of public 
issues. In hindsight, we wish we had made more 
inquiries about those lessons in our interviews. 
Fortunately, content specialists in other projects arc 
continuing to learn and document lessons (e.g.. Wolfe 
el al. l c ) c )3). Such individuals would benefit from 
increased opportunities to talk to and learn from one 
another about the common challenges of doing their 
work in the public arena. Process specialists could 
also help by talking with content specialists and 
compiling their lessons. 

4. Another topic receiving increased attention is linkages 
formed between educators and the news media to 
bring a more complex understanding of public issues 
to a larger audience. This was a workshop topic al the 
1993 National Public Policy Education Conference 
(Valenti 1994) and is a major priority of the Keitering 
Foundation. To our knowledge, however, actual 
experiences from which lo draw lessons continue to 
be I i mi led. 

5. The education-advocacy debate seems to have cooled. 
The topic is still discussed, of course, and warnings 
are issued regularly about the clangers of narrow 
advocacy. Simultaneously, the language of balance 
and fairness thai we suggested in Chapter 3 is used 
frequently (e.g., Dale 1993). Most importantly, people 
on both sides of the education-advocacy debate are 
now more likely to give reasons for their positions 
(e.g., Hite 1993: House 1993). They now bring lo 
more conscious and deliberate alien lion questions 
such as which stance is more ethical, which is more 
effective educationally, and what arc the implications 
of either stance for our organizations. YVc consider 
this a beneficial development. 

Impacts of education about public issues ( C hapter 
4) Much work remains lo be clone on ibis point. We 
think there is a tendency to report anecdotes more 
often — lo treat them a. c legitimate evidence — but not 
necessarily lo follow up such evidence with additional 
questions, as suggested in Chapter 4. We have only 
limited evidence of increased attention lo evaluation of 
education about public issues (Public Issues lidueation 
Materials Task Force 1904). There may be an increased 
tendency to include impacts on issues or on the process 
among project goals, bin serious efforts to evaluate such 
impacts have not yet come to our attention, li is possible 
thai relevant evaluation work is being done but simplv 
never gels widespread attention. Summarizing and 



drawing lessons from such evaluations would be a 
valuable contribution lo the practice of education about 
public issues. 

Given the magnitude of the problem with politics that 
we summarized at the beginning of this report, the 
efforts made by educators to date are only small steps. 
But they clearly seem to be sieps in the right direction. 
The)' are relevant and effective efforts lo address issues 
and problems of great importance in a democratic 
society. Valuable lessons can be learned from continued 
innovation and experimentation. Experiences and 
lessons need to be documented and shared with other 
educators, and the purposes, practice, and accomplish- 
ments of education about public issues heed to be 
communicated lo a larger audience of citizens and 
policymakers. Our study of ihc eleven Kellogg- and 
Farm Foundation-supported projects has left us even 
more convinced than we were at the outset about the 
importance of evaluation in developing and enhancing 
the capacity of educators to meet these challenges. 
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APPENDIX: METHODOLOGY 



This cluster evaluation used a ease study approach. 
Through methods that engaged us firsthand with the 
people, materials, and events of the eleven projects, we 
sought, first, to understand the story of each projects 
genesis, implementation, challenges, and successes. Second, 
vh an ongoing comparative analysis of these individual 
st iries, wc sought to extract broader understandings of 
pisntal elements of effective public policy education. Our 
overall purpose — derived from interactions with the W. K. 
Kellogg Foundation (WKKF) and the projects — was to 
probe for important lessons to be learned from this cluster 
of projects. 

EVALUATION QUESTIONS 

Wc derived the specific questions for our cluster evaluation 
from interactions with WKKF and project staff at the outset 
of our work, especially at the first cluster "networking 
conference" organized and sponsored by the foundation. 
These questions therefore reflect the combined interests of 
the foundation and the organizations and individuals 
participating in these eleven projects. The specific focus on 
coalitions represents primarily the foundation's interest in 
receiving substantive feedback on the meaningful ness and 
importance of coalitions for public policy education As will 
be recalled, this cluster was funded with a coalition require- 
ment. 

Wc used three basic questions to guide our cluster 
evaluation over the three years of the projects: 

1 . What is the nature, character, and meaningful ness of the 
coalitions formed in this public policy education cluster? 
In what ways did the coalitions relate to or affect project 
implementation and outcomes, including outcomes for 
individuals and for participating organizations and 
including project sustainability? Specifically, in what 
ways did project designs, implementations, or outcomes 
differ because of the coalition requirement? 

2. What were the outcomes of the projects? Specifically, 
what changes were demonstrated in the (a) knowledge, 
skills, or public policy involvement of individual project 
paitiapaiUs: (b) evolution of the policy issues addressed 
in the projects, e.g.. the infusion of a wider range of 
perspectives into the public debate; and (c) policy process 
itself, that is, who participates and how is diversity 
valued? 

5. \\ hat were the character and success ol the various 
educational strategies used in this cluster? Were there 



any important contextual factors for these projects, and, 
if so, what was the nature of their influence? Further, 
what were the linkages between project educational 
strategies and contextual factors, and project coalitions 
and outcomes? 

Befitting a case study approach, the meaning and sig- 
nificance of these questions evolved as issues of particular 
salience to these projects emerged. For example, within our 
emphasis on educational strategies, one focus became the 
nature and balance of projects' attention to issues of public 
policy content versus process. Similarly, within the coali- 
tions question, another focus became the rationale lor the 
formation of coalitions, or why those particular organiza- 
tions had come together to address a public policy issue. 

TEAM APPROACH 

Another major feature of our cluster evaluation was our 
team approach, which had two dimensions. 

The first represented our planned coordination with the 
project-level evaluators. In the original plan, responsibility 
and authority for collecting information related to the 
major evaluation questions were divided between the 
cluster evaluators and the project-level cvuluators. This was 
done to minimize duplication of effort and data collection 
burdens, while maximizing the value of the information to 
be collected for both the individual projects and the overall 
cluster. Specifically, the cluster evaluators were responsible 
lor addressing the coalitions evaluation question. Regarding 
the project outcomes question, project evaluators were to 
address unique outcomes. Common outcomes were to be 
addressed via a coordinated, cross-project data collection 
effort directed by the cluster evaluators working in collabo- 
ration with the project evaluators. Similarly, data collection 
on project implementation and context was envisioned as a 
shared responsibility. The working group for this cluster 
evaluation would be a team made up of the cluster evalua- 
tors and all project-level evaluators. 

In actuality, this team did not materialize, primarily 
because most of the eleven projects did not have a separate 
project-level evaluator. In most projects, one or more stall 
members also served as the project evaluators. This blur- 
ring ol program development and critique roles precluded a 
separate, distinctive emphasis on the latter in our own 
interactions with project stall. The major implications of 
this general absence of a strong evaluation presence at the 
project level concerned data collection on project out- 
comes, as discussed later. 
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The second dimension was a team of cluster evaluators. 
Our team had four members: one faculty member with 
expertise in public policy education and in extension 
education; one faculty member with expertise in program 
evaluation; and two doctoral students with developing 
expertise in evaluation and/or policy education. Three of 
the four team members remained w ith the cluster evalua- 
tion from start to finish, whereas one of the student slots 
turned over three times during the three-year inquiry 
period (July 1989 through August 1992). N4ost importantly, 
much of our analytic work was accomplished during our 
regular interactive team meetings. We intentionally used 
each other s perspectives, insights, and voices to try to 
understand what we were learning and to reflect on its 
public importance. 

A CASE STUDY APPROACH TO 
CLUSTER EVALUATION 

Our approach sought to understand each project s story 
and to use such understanding as the basis for cross-project 
analysis of major themes, patterns, and lessons learned. 

Overall Evaluation Design 

This case study approach is grounded in an interpret ivist 
logic of justification for evaluation (Cuba and Lincoln 
1989; Smith 1989). In this framework, emphasis is placed 
on what is meaningful to people in a given situation or 
context. Interpretivists assume that what is meaningful will 
vary across situations and across people within a situation. 
Diversity is valued, and multiple perspectives on and 
understandings of a given phenomenon are sought. These 
characteristics were well matched to the expected character 
of the public policy education projects in this cluster. 

At the outset of the cluster evaluation, teams ol one 
faculty member and one graduate student were formed for 
each project. The teams worked with that project through- 
out the inquiry period, enabling the development ol trust 
and openness in relationships between project staff and 
cluster evaluators. 

Data Collection 

We used three major forms of dala collection: site visits; 
colleciion and review of project documentation; and several 
more structured, supplementary data collection forms and 
surveys. 

Site visits. Teams made site visits to each project at 
least once during each of the three years of the cluster 
evaluation. Some projects were visital more than once each 
year. Iiach visit lasted two to lour days, the longer visits 
occurring toward the end ol the inquiry period. 
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During the first year, site visits were aimed at developing 
a solid project description. Toward that end, the teams 
interviewed project staff, coalition members, members of 
project advisory or management or materials development 
committees, people from the sponsoring organizations, and 
others important to the project s initiation and design. 
Project sites were visited and observations conducted. 
These observations contributed to understanding the 
projects context, design, initial implementation, challenges, 
and limitations. 

Site visits during the second year again included inter- 
views with project staff, coalition members, and other key 
project players, both new and old. These interviews focused 
on updating project descriptions and pursuing emerging 
themes of importance. These themes were selected in the 
year two networking conference of all project directors, 
sponsored by the WKKF. Some of the year two themes 
were also pursued by observing projects during si 1 visits. 
In particular, observations concentrated on (1) document- 
ing and gathering varied perceptions of actual project 
events (such as conferences, roundtahle discussions, 
training sessions, public forums, and "rivcrwalks") and (2) 
deepening our understanding of coalition dynamics and 
their relationships to decision making as manifested in staff, 
steering committee, or coalition meetings. 

Year three site visits were more tailored to each indi- 
vidual project. Project and coalition members were again 
interviewed, and a concerted effort was made to interview 
actual project participants to collect firsthand perceptions 
of the success and meaningfulness of project activities. Wc 
conducted both group interviews with representative 
groups of participants and telephone interviews with cross- 
sections ol participants. Certain project events or meetings 
were also observed. 

During all site visits, teams recorded each interview and 
observation in a data log. including the interviewer/ 
observers methodological and analytic comments. All logs 
were initially prepared by one team member and then 
reviewed for completeness and accuracy by I he other team 
member. These logs formed the basic raw data for the 
cluster evaluation. 

Collection and review of project documentation. 
The cluster evaluators tried, mostly successfully, to collect 
extensive documentation from each project. Specifically, we 
sought copies of all materials developed, minutes from 
project and coalition meetings, internal memos, relevant 
external correspondence, annual reports to WKKF, evalua- 
tion data and reports, news stories and press releases, and 
anything else of relevance to the evaluation questions, Such 
project documentation w as intended to lullill a largely 
descriptive and supportive role in the evaluation. Our 
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review of each project s documents aided substantially in 
developing a comprehensive descriptive portrait of each 
one. 

There are two limitations to this data collection strategy. 
First, we relied on project staff to share their documenta- 
tion. Most did, but in a few cases, we lacked backup 
substantiation of interview claims. Second, because project- 
level evaluators were not a separate group in this cluster, 
project-level evaluation data were uneven at best. In 
particular, our plans to work collaborativelv with project- 
level evaluators on the collection of common outcome data 
were not fulfilled. 

Supplementary data collection methods. During 
the second year, we collected additional data using three 
more structured methods. Each was designed to fill a 
perceived gap in our emerging understanding of projects, 
themes, and lessons. 

• An outcomes matrix was designed to address the 
absence of project-level outcome data. The rows of this 
matrix constituted four groups of project objectives — for 
coalition members, for participants, for the issue, and lor 
the policy process. These four groups reflected all 
written objectives in all project proposals. Space was also 
provided for objectives that emerged after the initiation 
of the project. The columns of the matrix requested 
project staff to identify which objectives were relevant to 
.their project and to provide or reference (as in an annual 
report) data relevant to the accomplishment of that 
objective. We had hoped that, given the organization of 
these data in a consistent format, we would be able to 
aggregate across projects and make some summary 
assessments. Unfortunately, the matrix did not work as 
designed. Although most project staff completed and 
returned it, they provided little actual data for us to 
aggregate. For some projects the matrix was premature 
because they did not yet have specific outcome informa- 
tion. Other projects were highly inconsistent in what 
they provided. Some were cautious and conservative in 
backing all their claims with specific evidence. Others 
were expansive and vague, offering only general refer- 
ences to data sources. 

The absence of credible data plagued other aspects of 
our cluster evaluation. Future public policy education 
clusters should be designed to avoid this problem. 

• A mailed survey ol involved individuals and groups. This 
survey evolved from a recognition that, in many projects, 
important contributions were being made by individuals 
without official status in or connections to the project, 
such as an informal set ol advisers. We believed that 
understanding the perceptions of these individuals 
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would help us develop a fuller picture of each project. 
Thus we elicited nominations of samples from project 
staff and designed and mailed a survey to these samples. 
The survey asked for respondents views on the nature 
and extent of their project involvement and on the 
project itself. A respectable response rate of about one- 
third was attained on this survey, and the results were 
used generally as intended. 

♦ A materials review process. This was conducted to elicit 
experts views of the quality and balance of the various 
materials developed by the projects. This effort reflects 
the tensions between education and advocacy in public 
policy education. We wanted expert opinion on the kind 
and degree of advocacy bias that might be present in the 
projects materials. A panel of three outside reviewers 
with diverse expertise was identified, a materials review 
form developed, samples (as needed) of materials 
selected, and a mailed review process implemented in 
which each set of materials was reviewed by two experts. 
The results of this external review were enlightening and 
extremely useful for our understanding of the issues 
involved. 

A separate summary of each of these more structured 
data collection efforts was also prepared. 

Data Analysis 

The data analyses in the cluster evaluation proceeded over 
the course of the evaluation from descriptive emphases to 
identification of themes to interpretations of key lessons 
learned. 

The data analysis took the same form each year. At the 
end ol the every year, descriptive case summaries were 
prepared for each project, integrating all the data collected 
for that project that year. For the second and third years, 
this involved updating the prior years descriptive sum- 
mary. Then, each year a cross-project analysis was con- 
ducted. For year one, this analysis followed an inductive, 
category-generation process and yielded a comprehensive 
set of categories for the cluster (e.g., "rationale for coalition 
formation" and "materials emphasis in project design"). In 
the second year, the analysis focused on selected analytic 
themes, developed via analytic memos. In the third and 
final year, the analysis focused on assessing the nature and 
quality of relationships among key cluster themes. 

It was advantageous for the cluster evaluation team that 
the annual Farm Foundation public policy education 
conference was held in September, just as our project years 
were ending. During years one and especially two of the 
cluster evaluation, one evaluator took the leadership to 
prepare a paper lor this conference that offered a reflective 




status report of our work. Other team members contributed 
reviews and commentaries. This opportunity to reflect on 
our work contributed to a more thoughtful and useful set of 
insights in the end. The only risk was that of becoming so 
enamored with emerging insights that they constrained or 
limited future information gathering and reflection on these 
projects. 

Data Quality 

We are confident that our work meets the relevant quality 
criteria for interprclivist case study evaluation offered by 
Cuba and Lincoln (1989). Specifically, wc believe our 
findings are ( i ) credible in that they are perceived as 
appropriate and accurate representations of actual project 
experiences and meanings by project people; (2) confivm- 
ahie in that they represent the voices we heard, in harmony 
with — not dominated by — our own views and values; and 
(3) dependable in that they emanate from professionally 
sound and defensible methodological decisions. 

We satisfied these inquiry criteria primaril) via ongoing 
internal team collaboration and external communication 
with project and coalition people. Internally, we made 
inquiry decisions, developed insights, probed emerging 
themes, formulated lessons — all collaboratively with 
intentional challenge, critique, and review. In parallel 
fashion, we shared our work with all project personnel, 
repeatedly inviting and receiving their critique, reflections, 
and insights. 

Reporting 

Major vehicles for sharing our insights, ideas, and decisions 
with project and foundation staff included writing regular 
progress repot is (three to four times a year), offering 
descriptive case summaries and data logs to interested 
project staff, routinely sharing copies of our reports and 
papers, and developing special reports and items for 
discussion at the annual networking meetings organized by 
the foundation. 



CONFIDENTIALITY 

All data were collected using approved procedures for 
protecting the privacy of the human participants. Specifi- 
cally, all respondents were offered full information about 
the evaluation and their participation in it, were asked for 
their voluntary participation, and were ensured that their 
individual responses would be kept wholly confidential. 

At the stage of reporting, all project directors were 
contacted and asked for permission to use actual organiza- 
tional names in this report. Permission was granted from all 
eleven projects. Individual identities, however, remain 
confidential. In this report, all individual names mentioned 
are pseudonyms, and most are a composite of several 
individuals. 
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